PanDA Native Workflow

A workflow is a set of tasks whose relationship is described with a directed acyclic graph (DAG), where each edge is directed from a parent task to a child task that processes the output data of the parent.

The PanDA native workflow is an implementation of workflows that runs entirely inside the PanDA server. The user describes the workflow in a single yaml file, the workflow description (WFD), and submits it with pchain_native. The server parses the description, registers the workflow, its steps and its data in the PanDA database, and drives the execution with its own workflow engine.

The native description language is deliberately close to the prun command line, so that a step is essentially one prun invocation plus the wiring that says where its input comes from. Currently steps are PanDA tasks submitted through prun, or sub-workflows.

Remark: the native workflow is a recent addition and is still evolving. The description language documented on this page covers plain DAGs, nested sub-workflows and scatter.

Quick start 

Write a workflow description, e.g. my_chain.yaml, and submit it from the directory that contains it together with any script that the steps execute.

$ pchain_native --wfd my_chain.yaml --outDS user.<your_nickname>.blah

$ pchain_native --wfd my_chain.yaml --outDS user.<your_nickname>.blah \
   --vo wlcg --prodSourceLabel test --workingGroup ${PANDA_AUTH_VO}

pchain_native packs all files under the current directory that are smaller than --maxSizeInSandbox (1 MB by default) into a sandbox, uploads it, and sends the workflow request to the PanDA server. The workflow description itself, any yaml file it references, and the executables that the steps run must therefore live in that directory tree.

--outDS is the basename of the datasets for output and log files. On success the server replies with the workflow_id that identifies the workflow.

To see all options of pchain_native

pchain_native --helpGroup ALL

The workflow description 

A workflow description is a yaml file with the following top-level sections.

Section	Description
name	Name of the workflow, shown in monitoring. Optional
inputs	Named input data of the workflow. Each entry is a dataset name, or a list of dataset names when it is used for scatter. Can be empty
outputs	Named final output data of the workflow. Each entry has a `from` field pointing to the output of a step, and an optional `output_types` list
steps	The steps of the workflow, i.e. the nodes of the DAG. Mandatory
options	Workflow-level options. Optional
workflow_blocks	Named sub-workflow definitions that steps in the same file can reference. Optional

Steps are given as a mapping from an arbitrary step name to the step definition. The type of a step is given in the type field and defaults to prun. A prun step takes the following fields.

Field	Corresponding prun option
inDS	—inDS (string)
inDsType	No correspondence. Type of inDS (string)
secondaryDSs	—secondaryDSs (a list of strings)
secondaryDsTypes	No correspondence. Types of secondaryDSs (a list of strings)
containerImage	—containerImage (string)
useAthenaPackages	—useAthenaPackages (bool)
args	all other prun options except for those listed in this table (string)
exec	—exec (string)

Essentially,

steps:
  top:
    type: prun
    args: "--outputs seed.txt --nJobs 3"
    exec: "echo %RNDM:10 > seed.txt"

corresponds to

prun --outputs seed.txt --nJobs 3 --exec "echo %RNDM:10 > seed.txt"

The usual prun placeholders are available. %IN in exec and args is expanded to the list of filenames in the input dataset given in inDS, %IN2, %IN3, … to the filenames of the secondary inputs, %{DSn} to the name of the n-th input dataset, %{SECDSn} to the name of the n-th secondary dataset, and %RNDM:n to a random number.

Wiring steps together 

There are only two reference forms in the language, and they are what turns a list of steps into a DAG.

Reference	Meaning
{name}	The workflow input name declared in the `inputs` section
step/outDS	The output data of the step step

For example

inputs:
  raw: mc16_valid:mc16_valid.900248.PG_singlepion_flatPt2to50.simul.HITS.e8312_s3238_tid26378578_00

steps:
  first:
    inDS: "{raw}"
    ...
  second:
    inDS: first/outDS
    ...

makes second a child of first, since second consumes what first produces. Note that {raw} has to be quoted, since a bare { starts a flow mapping in yaml.

The same references are used in secondaryDSs and in the from field of the workflow outputs. A step is a tail of the workflow when one of the workflow outputs refers to it.

A step has at most one primary input, given in inDS, and the task is split over its files so that each job gets a slice of them, reachable as %IN in exec. Everything listed in secondaryDSs is a secondary input: it does not drive the splitting, each job gets the number of files declared for it, and they are reachable as %IN2, %IN3, and so on. Both kinds make the source step a parent, so the child waits for either.

If a parent step produces several types of output data and the child needs only some of them, the types are selected with inDsType for the primary input and with secondaryDsTypes for the secondary inputs, positionally matching the entries of secondaryDSs. The stream name, the number of files per job, etc, for each secondary input are given with —secondaryDSs in args, where %{SECDSn} is the placeholder for the n-th secondary dataset name.

Reading the pictures 

The workflow DAGs on this page all use the same conventions.

In the picture	Meaning
Rounded box	A step, labelled with its name and the output types it produces
Folder, blue at the top / green at the bottom	An entry of the workflow `inputs` / `outputs` section
Solid arrow	A primary input: the target step names the source in its `inDS`
Dashed arrow	A secondary input: the target step lists the source in its `secondaryDSs`
Label on an arrow	The output type the child selects, i.e. its `inDsType` or `secondaryDsTypes` entry
Enclosing box	A sub-workflow, labelled with the name of the step that runs it
Enclosing boxes marked s1, s2, …	The iterations a scattered sub-workflow expands into, one per element of the scattered input lists

Where a sub-workflow has its own picture elsewhere, the enclosing box shows only the shape of it, with the text of its steps and inputs left out.

Output dataset names 

Each step is assigned a sequential number within its workflow, in topological order, and the output dataset of a step is named after the combination of --outDS, that number and the step name. The actual dataset in DDM appends the output file type, i.e.

<outDS>_<NNN>_<step_name>_<output_type>

For example, with —outDS user.<your_nickname>.blah the top step of the simple chain below writes to user.<your_nickname>.blah_001_top_intermediate.txt. If —outputs in args is a comma-separated list, one dataset is created for each output type. The naming of steps inside sub-workflows is described in the corresponding sections below.

Data availability and partial inputs 

A child step starts once the data it consumes are considered available. By default the workflow engine waits until the parent step is done, or until all output collections of the parent are closed and non-empty, so that the child sees the complete input.

That behaviour can be relaxed per workflow in the options section, so that a child starts while its parent is still running and both run in parallel for a while, which reduces the total execution time of the workflow.

Option	Default	Description
allow_partial_inputs	false	Let a step start when its input data are only partially available
min_input_files	10	With `allow_partial_inputs`, the minimum number of files required before a step starts

options:
  allow_partial_inputs: true
  min_input_files: 50

Workflow examples 

Simple chain 

The following description is a parent-child chain of two prun tasks.

simple_chain.yaml

# example of a simple chain workflow using the native YAML workflow description language

name: simple_chain  # optional; the workflow name shown in monitoring

inputs:
  input: "mc16_valid:mc16_valid.900248.PG_singlepion_flatPt2to50.simul.HITS.e8312_s3238_tid26378578_00"

outputs:
  final_output:
    from: bottom/outDS
    output_types:
      - results.root

steps:
  top:
    type: prun
    inDS: "{input}"
    args: "--outputs intermediate.txt --avoidVP"
    exec: "echo %IN > intermediate.txt"

  bottom:
    type: prun
    inDS: top/outDS
    args: "--outputs results.root --forceStaged --avoidVP"
    exec: "echo %IN > results.root"

The workflow takes one input, input, which is consumed by the top step through the {input} reference. bottom chains on the output of top with the step/outDS reference, which is what makes it a child of top. The workflow output final_output points at the output of bottom and declares results.root as its type, so bottom is the tail of the workflow.

Both steps are plain prun invocations, with %IN expanded to the filenames of the input each job gets. bottom starts processing once top has produced enough output data, waits if everything currently available has been processed while top is still running, and finishes once all data from top is processed.

Submit it with

pchain_native --wfd simple_chain.yaml --outDS user.<your_nickname>.blah

which produces user.<your_nickname>.blah_001_top_intermediate.txt and user.<your_nickname>.blah_002_bottom_results.root.

More complicated chain 

Steps can have several parents and several children, and the description below shows a workflow with two independent branches merging into a final step.

../_images/pchain_native_dag_signal_background_combine_wfd.png

signal_background_combine_wfd.yaml

# Signal and background processing workflow translated from CWL.
# Two independent branches (make_signal, make_background_1) feed into premix,
# while a separate generate_some branch feeds make_background_2; all streams
# are combined in the final combine step.
#
# Fill in the input dataset names before submitting.

name: signal_background_combine

inputs:
  signal: "mc16_valid:mc16_valid.900248.PG_singlepion_flatPt2to50.simul.HITS.e8312_s3238_tid26378578_00"
  background: "mc16_5TeV.361238.Pythia8EvtGen_A3NNPDF23LO_minbias_inelastic_low.merge.HITS.e6446_s3238_s3250/"

outputs:
  outDS:
    from: combine/outDS
    output_types:
      - aaa.root

steps:
  make_signal:
    type: prun
    inDS: "{signal}"
    containerImage: docker://busybox
    args: "--outputs abc.dat,def.zip --nFilesPerJob 5"
    exec: "echo %IN > abc.dat; echo 123 > def.zip"

  make_background_1:
    type: prun
    inDS: "{background}"
    args: "--outputs opq.root,xyz.pool --nGBPerJob 10"
    exec: "echo %IN > opq.root; echo %IN > xyz.pool"

  generate_some:
    type: prun
    args: "--outputs gen.root --nJobs 10"
    exec: "echo %RNDM:10 > gen.root"

  premix:
    type: prun
    inDS: make_signal/outDS
    inDsType: def.zip
    secondaryDSs:
      - make_background_1/outDS
    secondaryDsTypes:
      - xyz.pool
    args: "--outputs klm.root --secondaryDSs IN2:13:%{SECDS1}"
    exec: "echo %IN %IN2 > klm.root"

  make_background_2:
    type: prun
    inDS: "{background}"
    containerImage: docker://alpine
    secondaryDSs:
      - generate_some/outDS
    secondaryDsTypes:
      - gen.root
    args: "--outputs ooo.root,jjj.txt --secondaryDSs IN2:10:%{SECDS1}"
    exec: "echo %IN > ooo.root; echo %IN2 > jjj.txt"

  combine:
    type: prun
    inDS: make_signal/outDS
    inDsType: abc.dat
    secondaryDSs:
      - premix/outDS
      - make_background_2/outDS
    secondaryDsTypes:
      - klm.root
      - ooo.root
    args: "--outputs aaa.root --secondaryDSs IN2:2:%{SECDS1},IN3:5:%{SECDS2}"
    exec: "echo %IN %IN2 %IN3 > aaa.root"

The workflow takes two inputs, signal and background. The signal is used as input for the make_signal step, while the background is used as input for the make_background_1 and make_background_2 steps. generate_some has no input at all and only generates data.

make_signal runs in the busybox container given in containerImage and produces two types of output data, abc.dat and def.zip, as declared in —outputs. Its children pick the type they need: premix takes def.zip through inDsType, while combine takes abc.dat.

premix additionally reads xyz.pool from make_background_1 as a secondary input, and combine reads the outputs of both premix and make_background_2 as secondary inputs. Since combine is referenced by the workflow output, it is the tail of the workflow.

To run the same workflow over different input data it is enough to edit the inputs section and resubmit.

Nested workflow 

A workflow can be used as a step of another workflow. Such a step has workflow in its type field, and the workflow it runs is either referenced by file name or written inline.

The reference form points at another yaml file in the sandbox with workflow_ref.

In the picture the sub-workflow is drawn as structure only, since its steps are the ones already shown in the previous section.

../_images/pchain_native_dag_nested_workflow_sig_bg_comb_wfd.png

nested_workflow_sig_bg_comb_wfd.yaml

# Reference-based sub-workflow: the sig_bg_comb step runs the referenced
# signal_background_combine_wfd.yaml once, using that file's own declared inputs
# (signal/background). top produces a seed, and bottom consumes the sub-workflow
# output plus top's seed.

name: nested_workflow_sig_bg_comb

inputs: []

outputs:
  final_result:
    from: bottom/outDS
    output_types:
      - results.root

steps:
  top:
    type: prun
    args: --outputs seed.txt --nJobs 3 --avoidVP
    exec: "echo %RNDM:10 > seed.txt"

  sig_bg_comb:
    type: workflow
    workflow_ref: signal_background_combine_wfd.yaml   # <-- path relative to sandbox root

  bottom:
    type: prun
    inDS: sig_bg_comb/outDS
    secondaryDSs:
      - top/outDS
    secondaryDsTypes:
      - seed.txt
    args: --outputs results.root --forceStaged --avoidVP
    exec: "echo %IN %IN2 > results.root"

The sig_bg_comb step runs the whole signal_background_combine_wfd.yaml shown in the previous section once, using the inputs declared in that file. Both files must be in the sandbox, i.e. next to each other in the submission directory. workflow_ref can also name an entry of a workflow_blocks section in the same file, if you prefer to keep everything in one place.

Although a sub-workflow step runs many tasks, the rest of the description treats it exactly like an ordinary step: it has a single output, sig_bg_comb/outDS, which downstream steps consume. Under the hood, the outputs of the tail steps of the sub-workflow are aggregated into a DDM container named after the sub-workflow step, and that container is what bottom reads.

The same workflow can be written with the sub-workflow inlined in the steps field of the sub-workflow step, which then carries its own inputs, outputs and steps sections.

nested_workflow_inline_sig_bg_comb_wfd.yaml

# Same workflow as nested_workflow_sig_bg_comb_wfd.yaml, but the sub-workflow is
# expressed inline instead of via workflow_ref. The inline steps are identical to
# signal_background_combine_wfd.yaml.

name: nested_workflow_inline_sig_bg_comb

inputs: []

outputs:
  final_result:
    from: bottom/outDS
    output_types:
      - results.root

steps:
  top:
    type: prun
    args: --outputs seed.txt --nJobs 3 --avoidVP
    exec: "echo %RNDM:10 > seed.txt"

  sig_bg_comb:
    type: workflow
    inputs:
      signal: "mc16_valid:mc16_valid.900248.PG_singlepion_flatPt2to50.simul.HITS.e8312_s3238_tid26378578_00"
      background: "mc16_5TeV.361238.Pythia8EvtGen_A3NNPDF23LO_minbias_inelastic_low.merge.HITS.e6446_s3238_s3250/"
    outputs:
      outDS:
        from: combine/outDS
        output_types:
          - aaa.root
    steps:
      make_signal:
        type: prun
        inDS: "{signal}"
        containerImage: docker://busybox
        args: "--outputs abc.dat,def.zip --nFilesPerJob 5"
        exec: "echo %IN > abc.dat; echo 123 > def.zip"

      make_background_1:
        type: prun
        inDS: "{background}"
        args: "--outputs opq.root,xyz.pool --nGBPerJob 10"
        exec: "echo %IN > opq.root; echo %IN > xyz.pool"

      generate_some:
        type: prun
        args: "--outputs gen.root --nJobs 10"
        exec: "echo %RNDM:10 > gen.root"

      premix:
        type: prun
        inDS: make_signal/outDS
        inDsType: def.zip
        secondaryDSs:
          - make_background_1/outDS
        secondaryDsTypes:
          - xyz.pool
        args: "--outputs klm.root --secondaryDSs IN2:13:%{SECDS1}"
        exec: "echo %IN %IN2 > klm.root"

      make_background_2:
        type: prun
        inDS: "{background}"
        containerImage: docker://alpine
        secondaryDSs:
          - generate_some/outDS
        secondaryDsTypes:
          - gen.root
        args: "--outputs ooo.root,jjj.txt --secondaryDSs IN2:10:%{SECDS1}"
        exec: "echo %IN > ooo.root; echo %IN2 > jjj.txt"

      combine:
        type: prun
        inDS: make_signal/outDS
        inDsType: abc.dat
        secondaryDSs:
          - premix/outDS
          - make_background_2/outDS
        secondaryDsTypes:
          - klm.root
          - ooo.root
        args: "--outputs aaa.root --secondaryDSs IN2:2:%{SECDS1},IN3:5:%{SECDS2}"
        exec: "echo %IN %IN2 %IN3 > aaa.root"

  bottom:
    type: prun
    inDS: sig_bg_comb/outDS
    secondaryDSs:
      - top/outDS
    secondaryDsTypes:
      - seed.txt
    args: --outputs results.root --forceStaged --avoidVP
    exec: "echo %IN %IN2 > results.root"

The two forms are equivalent and give the DAG pictured above. A separate file is reusable across workflows and keeps the parent description short, while the inline form keeps a one-off sub-workflow self-contained in a single file.

References inside a sub-workflow are resolved in the scope of the sub-workflow, so {signal} binds to the inputs of the sub-workflow, not to those of the parent, and make_signal/outDS refers to the sub-workflow step of that name. The numbering of output datasets follows the nesting, i.e. the steps of the sub-workflow sig_bg_comb, which is step 002 of the outer workflow, write to

user.<your_nickname>.blah_002_001_make_signal_abc.dat
user.<your_nickname>.blah_002_002_make_background_1_opq.root
...

and the aggregated output of the sub-workflow step itself is user.<your_nickname>.blah_002_sig_bg_comb_aaa.root.

Sub-workflows can themselves contain sub-workflows, each level adding a segment to the name.

Keep the nesting shallow, up to about 5 levels. Deeper nesting is neither tested nor supported, and a limit is going to be enforced in a future version.

Scatter workflow 

A popular use-case is to perform the same analysis chain on many samples in a single workflow. This is expressed by scattering a sub-workflow over lists of inputs: the sub-workflow is instantiated once per element, and all the instances run in parallel. The description below holds a single sub-workflow, but the two-element input lists make it run twice, so the picture shows both instances, s1 and s2, each taking one element of each list.

../_images/pchain_native_dag_scatter_sig_bg_comb_wfd.png

scatter_sig_bg_comb_wfd.yaml

# Scatter sub-workflow: the referenced signal_background_combine_wfd.yaml is instantiated
# once per scattered item. scatter_inputs maps the parent's signals/backgrounds lists onto
# the sub-workflow's signal/background inputs, and scatter_mode: zip pairs them element-wise
# (signals[i] with backgrounds[i]) -- so here two sub-workflow instances run.
# Unlike an ordinary sub-workflow, the parent's scatter inputs replace the corresponding
# child inputs per iteration. The merge step then combines all per-iteration outputs.

name: scatter_sig_bg_comb

inputs:
  signals:
    - mc16_valid:mc16_valid.900248.PG_singlepion_flatPt2to50.simul.HITS.e8312_s3238_tid26378578_00
    - valid1.110910.Pythia8_AU2MSTW2008LO_zprime3000_tt.simul.HITS.e3365_s2112_tid04708744_00
  backgrounds:
    - mc16_5TeV.361238.Pythia8EvtGen_A3NNPDF23LO_minbias_inelastic_low.merge.HITS.e6446_s3238_s3250/
    - mc16_5TeV:mc16_5TeV.361239.Pythia8EvtGen_A3NNPDF23LO_minbias_inelastic_high.merge.HITS.e6446_s3238_s3250/

outputs:
  final_result:
    from: merge/outDS
    output_types:
      - merged.root

steps:
  many_sig_bg_comb:
    type: workflow
    workflow_ref: signal_background_combine_wfd.yaml   # <-- path relative to the sandbox root
    scatter_inputs:
      signal: signals
      background: backgrounds
    scatter_mode: zip

  merge:
    type: prun
    inDS: many_sig_bg_comb/outDS
    args: --outputs merged.root
    exec: "merge.sh merged.root %IN"

The workflow declares two lists, signals and backgrounds, in its inputs section. The many_sig_bg_comb step references the same signal_background_combine_wfd.yaml as before, and its scatter_inputs field maps the inputs of that sub-workflow onto those lists

scatter_inputs:
  signal: signals          # sub-workflow input : parent input list
  background: backgrounds
scatter_mode: zip

scatter_mode is zip, which pairs the lists element-wise, i.e. iteration i runs the sub-workflow with signals[i] and backgrounds[i]. Two instances therefore run here. zip is currently the only supported mode; if the lists have different lengths, the shortest one determines the number of iterations. Contrary to an ordinary sub-workflow, the values coming from the parent replace the inputs declared in the sub-workflow file, which are then unused.

The output of a scatter step aggregates the outputs of all its iterations, so the downstream merge step, which reads many_sig_bg_comb/outDS, sees the results of every sample and produces the final output.

Output datasets of the iterations carry both the number of the scatter step and the index of the iteration, e.g. the make_signal step of the second iteration writes to user.<your_nickname>.blah_001s2_001_make_signal_abc.dat.

Using workflow templates 

Some workflows are so regular that writing the description by hand is pure boilerplate. For those, pchain_native ships templates that generate the workflow description on the fly, so that the user only has to name the template and give its parameters.

pchain_native --template <template_name> --inDS <input_dataset> --outDS user.<your_nickname>.blah

--template is mutually exclusive with --wfd. The generated description is written to a temporary yaml file in the current directory, packed into the sandbox, and removed afterwards. Run with -v to see the name of the generated file and what the template decided.

Template parameters are passed as space-separated key=value pairs with --prunFlags, and are forwarded as —key value to every prun step of the generated workflow. Which keys are meaningful depends on the template. --prunFlags is optional, and so is each key within it: a template falls back to its own default for anything left out.

The multistep_merge template 

multistep_merge reduces a dataset to a single output file. A single merge job can only take so many files, so the reduction is done by a chain of prun merge steps, each one merging the output of its predecessor until a single file is left.

There is no workflow description to write: the whole chain is generated from one command line.

pchain_native --template multistep_merge --inDS <input_dataset> --outDS user.<your_nickname>.blah

The number of steps is not something you specify. The template queries Rucio for the number of files in --inDS and adds as many steps as the chain needs to converge, given how many files a single job merges. Run with -v to see what it found and how many steps it decided on.

This template requires no --prunFlags; both keys it understands have a default. Give them only to override how much a single merge job takes.

Parameter	Default	Description
nGBPerJob	10	Input size per job
maxNFilesPerJob	200	Maximum number of files merged by a single job. Also drives the number of steps

pchain_native --template multistep_merge --inDS <input_dataset> --outDS user.<your_nickname>.blah \
   --prunFlags nGBPerJob=10 maxNFilesPerJob=50

Each step runs merge.sh, a hadd-based script shipped with panda-client, which the template copies into the current directory so that it ends up in the sandbox. If a file with that name is already there, it is kept, so you can drop in your own merge script without touching the template.

The template needs the rucio-clients package to be importable in order to count the files of the input dataset.

What the template generates 

You do not need to read this to use the template, but it may be useful if you want to adapt what it produces. For an input dataset that takes three steps to reduce, the command above is equivalent to submitting the following description by hand.

multistep_merge_wfd.yaml

# example of a multi-step merge workflow using the native YAML workflow description language

name: multistep_merge_chain  # optional; the workflow name shown in monitoring

inputs:
  input_to_merge: "user.sgaid:user.sgaid.periodAllYear.physics_Main.DAOD_PHYS.grp23_v01_p6700.CNF_EMuJSMu_B_7cbc64d_output"

outputs:
  final_output:
    from: third/outDS
    output_types:
      - merge.root

steps:
  first:
    type: prun
    inDS: "{input_to_merge}"
    args: >-
      --outputs merge.root --rootVer recommended --noBuild --notExpandInDS
      --nGBPerJob 10 --maxNFilesPerJob 50 --respectSplitRule
      --writeInputToTxt IN:input.lis --avoidVP
    exec: merge.sh

  second:
    type: prun
    inDS: first/outDS
    args: >-
      --outputs merge.root --rootVer recommended --noBuild --notExpandInDS
      --nGBPerJob 10 --maxNFilesPerJob 50 --respectSplitRule
      --writeInputToTxt IN:input.lis --avoidVP
    exec: merge.sh

  third:
    type: prun
    inDS: second/outDS
    args: >-
      --outputs merge.root --rootVer recommended --noBuild --notExpandInDS
      --nGBPerJob 10 --maxNFilesPerJob 50 --respectSplitRule
      --writeInputToTxt IN:input.lis --avoidVP
    exec: merge.sh

options:
  # specify any workflow-level options here
  # allow_partial_inputs: true  # allow steps to run even if inputs are not complete yet
  # min_input_files: 50  # for partial inputs, require at least 50 files before starting steps

Every step is the same prun invocation, differing only in where its input comes from. Since —writeInputToTxt is given, prun writes the list of input files of each job to input.lis and merge.sh merges the files listed there. The --prunFlags of the template land in args, which is why they apply to every step.

Checking before submission 

Workflow descriptions can be error-prone, and a mistake in a reference or in a prun option is only reported once the workflow has been parsed on the server. --noSubmit performs everything that submission does except contacting the server, i.e. it builds the sandbox and the task parameters, so that local problems surface immediately.

pchain_native --wfd my_chain.yaml --outDS user.<your_nickname>.blah --noSubmit -v

Once the workflow is submitted, the server parses the description, dumps the internally converted node list to the server log, and cancels the workflow if a step has unresolved inputs, an unknown field, or a placeholder that cannot be expanded.

Monitoring 

pchain_native reports the workflow_id assigned by the server on successful submission. The workflow, its steps and the tasks they submit can be followed in PanDA monitoring, and the tasks themselves are ordinary PanDA tasks named after their output datasets, i.e. <outDS>_<NNN>_<step_name>.