parsl.data_provider.ftp.FTPSeparateTaskStaging

class parsl.data_provider.ftp.FTPSeparateTaskStaging[source]

Performs FTP staging as a separate parsl level task.

__init__()[source]

Methods

__init__()

can_stage_in(file)

Given a File object, decide if this staging provider can stage the file.

can_stage_out(file)

Like can_stage_in, but for staging out.

replace_task(dm, executor, file, func)

For a file to be staged in, optionally return a replacement app function, which usually should be the original app function wrapped in staging code.

replace_task_stage_out(dm, executor, file, func)

For a file to be staged out, optionally return a replacement app function, which usually should be the original app function wrapped in staging code.

stage_in(dm, executor, file, parent_fut)

This call gives the staging provider an opportunity to prepare for stage-in and to launch arbitrary tasks which must complete as part of stage-in.

stage_out(dm, executor, file, app_fu)

This call gives the staging provider an opportunity to prepare for stage-out and to launch arbitrary tasks which must complete as part of stage-out.

can_stage_in(file)[source]

Given a File object, decide if this staging provider can stage the file. Usually this is be based on the URL scheme, but does not have to be. If this returns True, then other methods of this Staging object will be called to perform the staging.

stage_in(dm, executor, file, parent_fut)[source]

This call gives the staging provider an opportunity to prepare for stage-in and to launch arbitrary tasks which must complete as part of stage-in.

This call will be made with a fresh copy of the File that may be modified for the purposes of this particular staging operation, rather than the original application-provided File. This allows staging specific information (primarily localpath) to be set on the File without interfering with other stagings of the same File.

The call can return a:
  • DataFuture: the corresponding task input parameter will be replaced by the DataFuture, and the main task will not run until that DataFuture is complete. The DataFuture result should be the file object as passed in.

  • None: the corresponding task input parameter will be replaced by a suitable automatically generated replacement that container the File fresh copy, or is the fresh copy.