May 21 – 24, 2018
Fluno Center on the University of Wisconsin-Madison Campus
America/Chicago timezone

Pegasus + HTCondor DAGMan: Expanding Scientific Workflows Horizons

May 22, 2018, 4:35 PM
20m
Howard Auditorium (Fluno Center on the University of Wisconsin-Madison Campus)

Howard Auditorium

Fluno Center on the University of Wisconsin-Madison Campus

601 University Avenue, Madison, WI 53715-1035

Speaker

Karan Vahi (Pegasus - Team USC Information Sciences Institute)

Availability of the Speaker<br>Let us know if there are times you CANNOT present,<br>prehaps because you need to leave for the airport early, etc.

Preference is for Tuesday or Wednesday.

Summary (2-4 sentences)<br>Just a few informal sentences describing what you want to present.<br>No need to spend a lot of time on this! You can change it later.

Pegasus builds on HTCondor DAGMan to deliver a powerful workflow management system.  Pegasus enables researchers to describe their computing pipelines in a portable, resource agnostic description, and compiles the high level description to an executable form described as a HTCondor DAGMan. A variety of computational resources can be targeted. During the compilation process, Pegasus transforms the workflow to increase performance, reliability and introduces a variety of data management transformations such as data transfers and cleanup nodes to the workflow. Using Pegasus, researchers are able to track and monitor their workflows using a suite of command line tools and a web dashboard.

The talk will give an overview of the system, and also introduce recently released and upcoming features, such as application containers and data integrity checking.

Primary author

Karan Vahi (Pegasus - Team USC Information Sciences Institute)

Presentation materials