common

pilot.user.generic.common.allow_timefloor(submitmode)

Should the timefloor mechanism (multi-jobs) be allowed for the given submit mode?

Parameters:

submitmode – submit mode (string).

pilot.user.generic.common.get_analysis_run_command(job, trf_name)

Return the proper run command for the user job.

Example output: export X509_USER_PROXY=<..>;./runAthena <job parameters> –usePFCTurl –directIn

Parameters:

• job – job object.

• trf_name – name of the transform that will run the job (string). Used when containers are not used.

Returns:

command (string).

pilot.user.generic.common.get_metadata(workdir)

Return the metadata from file.

Parameters:

workdir – work directory (string)

Returns:

pilot.user.generic.common.get_payload_command(job)

Return the full command for executing the payload, including the sourcing of all setup files and setting of environment variables.

By default, the full payload command is assumed to be in the job.jobparams.

Parameters:

job – job object

Returns:

command (string)

pilot.user.generic.common.get_utility_command_execution_order(name)

Should the given utility command be executed before or after the payload?

Parameters:

name – utility name (string).

Returns:

execution order constant (UTILITY_BEFORE_PAYLOAD or UTILITY_AFTER_PAYLOAD_STARTED)

pilot.user.generic.common.get_utility_command_kill_signal(name)

Return the proper kill signal used to stop the utility command.

Parameters:

name

Returns:

kill signal

pilot.user.generic.common.get_utility_command_output_filename(name, selector=None)

Return the filename to the output of the utility command.

Parameters:

• name – utility name (string).

• selector – optional special conditions flag (boolean).

Returns:

filename (string).

pilot.user.generic.common.get_utility_command_setup(name, setup=None)

Return the proper setup for the given utility command. If a payload setup is specified :param name: :param setup: :return:

pilot.user.generic.common.get_utility_commands(order=None, job=None)

Return a dictionary of utility commands and arguments to be executed in parallel with the payload. This could e.g. be memory and network monitor commands. A separate function can be used to determine the corresponding command setups using the utility command name. If the optional order parameter is set, the function should return the list of corresponding commands. E.g. if order=UTILITY_BEFORE_PAYLOAD, the function should return all commands that are to be executed before the payload. If order=UTILITY_WITH_PAYLOAD, the corresponding commands will be prepended to the payload execution string. If order=UTILITY_AFTER_PAYLOAD_STARTED, the commands that should be executed after the payload has been started should be returned.

FORMAT: {‘command’: <command>, ‘args’: <args>}

Parameters:

• order – optional sorting order (see pilot.util.constants)

• job – optional job object.

Returns:

dictionary of utilities to be executed in parallel with the payload.

pilot.user.generic.common.post_prestagein_utility_command(**kwargs)

Execute any post pre-stage-in utility commands.

Parameters:

kwargs – kwargs (dictionary).

Returns:

pilot.user.generic.common.post_utility_command_action(name, job)

Perform post action for given utility command.

Parameters:

• name – name of utility command (string).

• job – job object.

Returns:

pilot.user.generic.common.process_debug_command(debug_command, pandaid)

In debug mode, the server can send a special debug command to the pilot via the updateJob backchannel. This function can be used to process that command, i.e. to identify a proper pid to debug (which is unknown to the server).

Parameters:

• debug_command – debug command (string), payload pid (int).

• pandaid – PanDA id (string).

Returns:

updated debug command (string)

pilot.user.generic.common.remove_redundant_files(workdir, outputfiles=[], islooping=False, debugmode=False)

Remove redundant files and directories prior to creating the log file.

Parameters:

• workdir – working directory (string).

• outputfiles – list of output files.

• islooping – looping job variable to make sure workDir is not removed in case of looping (boolean).

Returns:

pilot.user.generic.common.sanity_check()

Perform an initial sanity check before doing anything else in a given workflow. This function can be used to verify importing of modules that are otherwise used much later, but it is better to abort the pilot if a problem is discovered early.

Returns:

exit code (0 if all is ok, otherwise non-zero exit code).

pilot.user.generic.common.update_job_data(job)

This function can be used to update/add data to the job object. E.g. user specific information can be extracted from other job object fields. In the case of ATLAS, information is extracted from the metaData field and added to other job object fields.

Parameters:

job – job object

Returns:

pilot.user.generic.common.update_server(job)

Perform any user specific server actions.

E.g. this can be used to send special information to a logstash.

Parameters:

job – job object.

Returns:

pilot.user.generic.common.update_stagein(job)

In case special files need to be skipped during stage-in, the job.indata list can be updated here. See ATLAS code for an example.

Parameters:

job – job object.

Returns:

pilot.user.generic.common.validate(job)

Perform user specific payload/job validation.

Parameters:

job – job object.

Returns:

Boolean (True if validation is successful).

pilot.user.generic.common.verify_job(job)

Verify job parameters for specific errors. Note:

in case of problem, the function should set the corresponding pilot error code using job.piloterrorcodes, job.piloterrordiags = errors.add_error_code(error.get_error_code())

Parameters:

job – job object

Returns:

Boolean.