PanDA API (1.0.0)

Submit a grep request to be processed asynchronously on the target service or machine.

Submit a sleep+echo request, run on any one machine in the target service. Results are readable by the requester or any production-role caller (access="production").

Poll for the results of an async request, of any type and from any submitting module.

The response has two shapes, depending on what the request's handler stores.

Handlers writing raw output (grep, sleep_echo) report one entry per machine:

{
    "success": bool,        # whether this poll succeeded
    "message": str,
    "data": {
        "overall_status": "complete" | "pending",
        "expected_machines": [str, ...],
        "results": [{"machine_name": str, "status": str, "result": str,
                      "truncated": int, "error_msg": str, "attempts": int,
                      "started_at": str, "finished_at": str,
                      "stderr": str, "return_code": int}, ...]
    }
}

overall_status is "complete" when all expected machines have a terminal result (done/failed).

Handlers writing a structured payload (e.g. the Data Carousel operations submitted by pandaserver.api.v1.data_carousel_api) report that payload at the top level instead:

{
    "success": bool,        # whether the OPERATION succeeded
    "message": str,         # the operation's message
    "data": <the operation's data>,
    "async_meta": {"status": "pending" | "running" | "done" | "failed",
                   "attempts": int, "started_at": str, "finished_at": str,
                   "error_msg": str}
}

Poll on async_meta.status, not on success: success is False while the request is still pending or running, and False again when the handler crashed (status "failed", reason in async_meta.error_msg), so it only tells the operation's outcome once status is "done". async_meta is present whenever the poll itself succeeded, so a response without it is a failure of this call (not found, not authorized) rather than a report about the request.

Set a key-value pair to store in PanDA. Requires a secure connection.

Get the secrets for a user identified by a list of keys. Requires a secure connection.

This function retrieves the distinguished name (DN) from the request and uses it to get a key pair. Requires a secure connection.

Get the x509 proxy certificate for a user with a role. Requires a secure connection.

Get the OAuth access token for the specified client. Requires a secure connection.

This function retrieves the distinguished name (DN) from the request and uses it to get a token key for the specified client. Requires a secure connection.

Each operation comes in two flavours sharing one implementation (pandaserver.taskbuffer.data_carousel_ops):

• the synchronous endpoints (change_staging_destination, ...) run the operation inline and return its outcome, at the risk of hitting the HTTP timeout on slow DDM/iDDS calls;
• the asynchronous endpoints (submit_change_staging_destination, ...) register a request in the async_requests table and return immediately with an async_id; the async request watchdog on JEDI runs the very same operation and the outcome is polled with the shared reader /v1/async_process/get_result.

Both flavours are kept so callers can switch between them without a server-side change.

The current active staging request will be cancelled, and a new request will be created with the newly selected destination RSE, excluding the original destination. The requests can be specified by request_id or dataset (if both exist, request_id is taken). Requires a secure connection production role. Runs synchronously; see submit_change_staging_destination for the asynchronous flavour.

If the request is queued, its source_rse will be rechosen, excluding the original source. If the request is staging, the source_replica_expression of its DDM rule is unset so new source can be tried. Only effective on queued or staging requests. The requests can be specified by request_id or dataset (if both exist, request_id is taken). Requires a secure connection production role. Runs synchronously; see submit_change_staging_source for the asynchronous flavour.

The request will skip the queue and go to staging immediately (will submit DDM rules). Only effective on queued requests. The requests can be specified by request_id or dataset (if both exist, request_id is taken). Requires a secure connection production role. Runs synchronously; see submit_force_to_staging for the asynchronous flavour.

If the request is done and has no related tasks, it can be retired to clean up the DDM rules and replicas. The requests can be specified by request_id or dataset (if both exist, request_id is taken). Requires a secure connection production role. Runs synchronously; see submit_retire_unused for the asynchronous flavour.

Asynchronous flavour of change_staging_destination: the request is registered in DB and processed by the async request daemon, so the call returns without waiting for DDM or iDDS. Poll /v1/async_process/get_result with the returned async_id as its request_id to get the outcome; it reports the operation's own success, message and data at the top level. Requires a secure connection production role.

Asynchronous flavour of change_staging_source: the request is registered in DB and processed by the async request daemon, so the call returns without waiting for DDM or iDDS. Poll /v1/async_process/get_result with the returned async_id as its request_id to get the outcome; it reports the operation's own success, message and data at the top level. Requires a secure connection production role.

Asynchronous flavour of force_to_staging: the request is registered in DB and processed by the async request daemon, so the call returns without waiting for DDM or iDDS. Poll /v1/async_process/get_result with the returned async_id as its request_id to get the outcome; it reports the operation's own success, message and data at the top level. Requires a secure connection production role.

Asynchronous flavour of retire_unused: the request is registered in DB and processed by the async request daemon, so the call returns without waiting for DDM or iDDS. Poll /v1/async_process/get_result with the returned async_id as its request_id to get the outcome; it reports the operation's own success, message and data at the top level. Requires a secure connection production role.

This function returns the count of available event ranges for a given job_id, jobset_id, and task_id. Requires a secure connection and production role.

Gets a dictionary with the status of the event ranges for the given pairs of PanDA job IDs and JEDI task IDs. Requires a secure connection.

Acquires a list of event ranges with a given PandaID for execution. Requires a secure connection and production role.

Updates the status of a specific event range. Requires a secure connection and production role.

Updates the event ranges in bulk. Requires a secure connection and production role.

Uploads a JEDI log file and returns the URL to the file. If there is already a log file for the task, it will be overwritten. Requires a secure connection and production role.

Updates a JEDI log file, appending more content at the end of the file. Requires a secure connection and production role.

Downloads the JEDI log file, if required at a particular offset.

Uploads a file to the cache. When not touched, cache files are expired after some time. User caches will get registered in the PanDA database and will account towards user limits. PanDA log files will be stored in gzip format. Requires a secure connection.

Touches a file in the cache directory. It avoids the file to expire and being deleted by server clean up processes. Requires a secure connection.

Deletes a file from the cache directory. Currently a dummy method. Requires a secure connection.

Registers a file from the cache directory into the PanDA database, so that PanDA knows the server it's on. Requires a secure connection and production role.

Validates a cache file owned by the caller by checking the file metadata that was registered in the database. Requires a secure connection.

Uploads a HPO checkpoint file to the server. Requires a secure connection.

Deletes a HPO checkpoint file from the server. Requires a secure connection.

Upload request to recover lost files. Either task_id or dataset needs to be specified. Requires a secure connection.

Uploads a workflow request to the server. The request can be processed synchronously or asynchronously. Requires a secure connection.

Uploads an event picking request to the server. Requires a secure connection.

Update the details for a list of workers. Requires a secure connection.

Update the service metrics for a harvester instance. Requires a secure connection.

Add messages for a harvester instance. Requires a secure connection.

Send a heartbeat for harvester and optionally update the instance data. User and host are retrieved from the request object and updated in the database. Requires a secure connection.

Retrieve the current worker ID. Requires a secure connection.

Get statistics for all the workers managed across the Grid. Requires a secure connection.