HTCondor Week 2019

May 20, 2019, 8:00 AM → May 23, 2019, 2:00 PM America/Chicago

Discovery Building on the University of Wisconsin-Madison Campus

Brian Bockelman (University of Nebraska-Lincoln), Mark Coatsworth (University of Wisconsin), Todd Tannenbaum (University of Wisconsin)

Mon, May 20

8:00 AM Coffee and Registration

Monday AM Advanced Tutorials 1

Conveners: Brian Lin (UW-Madison CHTC), Mr John Knoeller (University of Wisconsin, Madison)

1 Philosophy of HTCondor

Speaker: Mr Greg Thain (Center for High Throughput Computing)

Slides Philosophy.pptx

2 ClassAd Advanced Tutorial

Speaker: Mr Jaime Frey (Center for High-Throughput Computing)

Slides CW19-classads.ppt

3 HTCondor in the Cloud

Speaker: Mr Todd Miller (CHTC)

Slides condor_annex-Tutorial-v7.pptx

Monday AM Introductury Tutorials 1

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

4 Introduction to CHTC

Speaker: Lauren Michael (UW-Madison)

Slides 2019HTCondorWeek_IntroCHTC_LMichael.pdf 2019HTCondorWeek_IntroCHTC_LMichael.pptx

5 Intro to HTCondor Job Submission

Speaker: Christina Koch (UW Madison)

Slides 2019-Koch-User-Tutorial.pdf 2019-Koch-User-Tutorial.pptx

10:20 AM Break

Monday AM Advanced Tutorials 2

Conveners: Ian Ross (U. Wisconsin), Mr Zach Miller (UW Madison)

6 Negotiator Policy and Configuration

Speaker: Mr Greg Thain (Center for High Throughput Computing)

Slides Negotiator.pptx

7 Advanced Submission Tutorial

Speaker: Mr John Knoeller (University of Wisconsin, Madison)

Slides TJs_Advanced_Submit_Condor_Week_2019.pptx

Monday AM Introductory Tutorials 2

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

8 HTCondor Job Testing, Troubleshooting, and Special Use Cases

Speaker: Christina Koch (UW Madison)

9 Manage Workflows with HTCondor’s DAGMan

Speaker: Lauren Michael (UW-Madison)

Slides 2019HTCondorWeek_AutomatedWorkflowsDAGMan_LMichael.pdf 2019HTCondorWeek_AutomatedWorkflowsDAGMan_LMichael.pptx

12:15 PM Lunch

Monday PM Advanced Workshops 1

Conveners: Aaron Moate (UW-Madison CHTC), Greg Thain (Center for High Throughput Computing)

10 Python Bindings Tutorial

Speaker: Brian Bockelman (University of Nebraska-Lincoln)

Slides Bockelman-HTCondorWeek-Python-2019.pdf http://bit.ly/htcpy-stable

11 Work Queue tutorial

Speaker: Dr Benjamin Tovar (University of Notre Dame)

Slides 05-20-2019-Work_Queue-@-CondorWeek.pdf

Monday PM CHTC Research 1

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

12 Inferring Effective Connectivity From High-Dimensional ECoG Recordings

Speaker: Chris Endemann (UW-Madison Department of Anesthesiology)

Slides HTCondorWeek2019_Endemann_EffectiveConnectivity.pdf HTCondorWeek2019_Endemann_EffectiveConnectivity.pptx

13 Genomic ancestry analysis in wild hybrid house mice

Speaker: Megan Frayer (UW-Madison Laboratory of Genetics)

Slides Frayer_HTCondorWeek_2019_web.pdf Frayer_HTCondorWeek_2019_web.pptx

14 Uncertainty propagation in Fuel Cycle simulation using HTC

Speaker: Baptiste Mouginot (UW-Madison Department of Engineering)

Slides Fuel_Cycle_simulation_using_HTC_Condor.pdf

15 Alignment and CNV analysis in Cattle

Speaker: Beth Lett (UW-Madison Department of Animal Science)

Slides BL_Presentation.pptx

2:45 PM Break

2:45 PM Break

Monday PM Advanced Workshops 2

Conveners: Carl Edquist (UW-Madison CHTC), Mark Coatsworth (University of Wisconsin)

16 Pegasus Workshop

Speaker: Mr Karan Vahi (Pegasus - Team USC Information Sciences Institute)

Slides pegasus-chtc-tutorial-final.pdf

Monday PM CHTC Research 2

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

17 Image Analysis and Computation-Based Phenotyping

Speaker: Nathan Miller (UW-Madison Department of Botany)

18 Grid Search Methods for Modeling Polymerization Kinetics

Speaker: Katie Ziebarth (UW-Madison Department of Chemistry)

Slides HTCondorWeek2019_KZiebarth_Final.pdf HTCondorWeek2019_KZiebarth_Final.pptx

19 Studying RNA Virus Replication with Cryo-Electron Microscopy on HTC

Speaker: Hong Zhan (Morgridge Institute for Research)

Slides HTC_presentation_HZ.pdf HTC_presentation_HZ.pptx

4:15 PM Ice Cream Mixer: talk with other HTCondor users

Tue, May 21

8:00 AM Coffee and Registration

Tuesday AM Welcome and Keynote

Conveners: Mark Coatsworth (University of Wisconsin), Mr Todd Tannenbaum (University of Wisconsin)

20 Welcome to HTCondor Week

Speaker: Miron Livny (UW-Madison CHTC)

Slides 05-19-CW.pptx

21 Ice Fishing for Neutrinos: Exploring the mysteries of the cosmos from beneath the South Pole

Speaker: Dr Francis Halzen (UW-Madison WIPAC)

22 Computer and Data Sciences at UW-Madison

Speaker: Remzi Arpaci-Dusseau (UW-Madison Department of Computer Sciences)

Slides CondorHTC-Spring19_(3).pptx

10:15 AM Break

Tuesday AM Technical Talks

Conveners: Aaron Moate (UW-Madison CHTC), Ian Ross (U. Wisconsin)

23 IceCube Computing

Speaker: Benedikt Riedel (UW-Madison WIPAC)

Slides HTCondorWeek_2019.pdf HTCondorWeek_2019.pptx

24 HTCondor in Foxconn Hyperscale AI HPC Cloud

Speaker: Hank Du (Foxconn Industrial Internet)

Slides HTCondor_DCT_20190411_v4.pdf

25 Orpheus: Efficient Distributed Machine Learning via System and Algorithm Co- design

Speaker: Pengtao Xie (Petuum Inc.)

26 What's new in HTCondor? What is upcoming?

Speaker: Mr Todd Tannenbaum (University of Wisconsin)

Slides TannenbaumT_WhatsNew_HTCondor_Week_2019.pptx

12:15 PM Lunch

Tuesday PM Technical Talks 1

Conveners: Mr Jaime Frey (Center for High-Throughput Computing), Tim Slauson (UW-Madison CHTC)

27 Space Telescope Science Institute (STScI) and HTCondor – Reporting on today and exploring tomorrow.

Speaker: Mrs Mary Romelfanger (Space Telescope Science Institute)

Slides HTCondor_2019_STScI.pdf HTCondor_2019_STScI.pptx

28 Simplifying submitting for our users with a custom submission API

Speaker: Mr Collin Mehring (Dreamworks Animation)

Slides HTCondorWeek_2019.pptx

29 Updates from LIGO

Speaker: James Clark (Georgia Institute of Technology (LIGO))

Slides JamesClarkCondorWeek.pdf

30 'Startd flocking' at and around the Milan Physics Dept.

Speaker: Francesco Prelz (INFN - Sezione di Milano)

Slides http://www.mi.infn.it/~prelz/condor_week_2019/

31 HTCondor in Astronomy at NCSA

Speakers: Greg Daues (NCSA), Michael Johnson (National Center for Supercomputing Applications)

Slides HTCondor_NCSA_2019.pdf HTCondor_NCSA_2019.pptx

3:20 PM Break

Tuesday PM Technical Talks 2

Conveners: Josh Karpel (UW-Madison CHTC), Matyas Selmeci (UW-Madison CHTC)

32 An opportunistic HTCondor pool inside a interactive-friendly Kubernetes cluster

Speaker: Mr Igor Sfiligoi (University of California San Diego)

Slides HTCondor_PRP_2019.pdf HTCondor_PRP_2019.pptx

33 Pegasus : Introducing Integrity to Scientific Workflows

Speaker: Mr Karan Vahi (Pegasus - Team USC Information Sciences Institute)

Slides vahi-pegasus-cw-2019-final.pdf vahi-pegasus-cw-2019-final.pptx

34 AI platform on HTcondor

Speaker: Mr Vishnu Vettrivel (Wisecube)

Slides ai_platform_htcondor_wisecube.pdf ai_platform_htcondor_wisecube.pptx

35 Tuesday Closing Remarks

Speaker: Miron Livny (UW-Madison CHTC)

Amazon Web Services Social

Wed, May 22

8:00 AM Coffee and Registration

Wednesday AM Campus Research

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

36 Facilitating High Throughput Computing

Speaker: Lauren Michael (UW-Madison)

Slides 2019HTCondorWeek_RCFacilitation_LMichael.pdf 2019HTCondorWeek_RCFacilitation_LMichael.pptx

37 Improving the quality of my research on urban residential hydrology by increasing the quantity of simulations using HTCondor

Speaker: Carolyn Voter (University of Wisconsin-Madison)

Slides 2019.05_htcondor_week_presentation_cvoter.pptx

38 Analyzing Soil Microbial Community Sequencing Data with the CHTC

Speaker: Thea Whitman (UW-Madison Department of Soil Science)

Slides CHTC_Whitman_2019.pdf

39 High Throughput Computing For the Generation of a Self-Stabilizing Laser Sail Capable of Exploring Beyond Our Solar System

Speaker: Joel Siegel (UW-Madison Department of Physics)

Slides PodHandler.pdf PodHandler.pptx

10:35 AM Break

Wednesday AM Technical Talks

Conveners: Tim Theisen (UW-Madison CHTC), Mr Todd Miller (CHTC)

40 High-throughput processing of telemetry data

Speakers: Mr Colton Mikes (Texas Tech University), Dr DJ Lee (Brigham Young University), Dr Richard Erickson (US Geological Survey)

Slides USGS_HTCondor-updated.pdf USGS_HTCondor-updated.pptx

41 Deploying and managing containers on OSG

Speaker: Marco Mambelli (Fermilab)

Slides GlideinWMS-condor_week2019v2.pdf

42 Scientific Computing using AWS and HTCondor

Speaker: Dr Sanjay Padhi (AWS)

Slides AWS_HTCondor2019_pub.pdf

12:15 PM Lunch

Wednesday PM Technical Talks 1

Conveners: Brian Lin (UW-Madison CHTC), Josh Karpel (UW-Madison CHTC)

43 HTCondor Solutions for the Future of the CMS Submission Infrastructure

Speaker: Dr JAMES LETTS (UCSD)

Slides JLETTS-CMSHTCondorWeek2019_.pdf

44 Managing a Dynamic Sharded Pool

Speaker: Anthony Tiradani (Fermilab National Accelerator Laboratory)

Slides FNAL_HTCondor_Week_2019.pdf FNAL_HTCondor_Week_2019.pptx

45 HTCondor with google Cloud Platform

Speaker: Michiru Kaneda (The University of Tokyo)

Slides 20190522_kaneda_htcondor_week.pdf 20190522_kaneda_htcondor_week.pptx

46 HTCondor Solutions for Several Scenarios at IHEP

Speaker: Jiaheng Zou (Institute of High Energy Physics (IHEP) of the Chinese Academy of Sciences (CAS))

Slides HTCondor_at_IHEP.pdf HTCondor_at_IHEP.pptx

47 Job Submission via Share Filesystem: Running HTCondor Jobs on Isolated HPC Machines

Speaker: Mr Jaime Frey (Center for High-Throughput Computing)

Slides CW19-FileSystemJobs.ppt

3:20 PM Break

Wednesday PM Technical Talks 2

Conveners: Jason Patton (University of Wisconsin), Tim Slauson (UW-Madison CHTC)

48 Python bindings based monitoring at CMS

Speaker: Diego Davila (CERN)

Slides python_bindings_monitoring.pdf

49 Thinking about HTC Notebooks: High Throughput Computing via Jupyter Notebooks

Speaker: Mr Todd Tannenbaum (University of Wisconsin)

Slides TannenbaumT_HTC_Notebooks.pdf TannenbaumT_HTC_Notebooks.pptx

50 HTMap: High-Throughput Computing in Python

Speaker: Mr Josh Karpel (University of Wisconsin - Madison)

Slides HTMap.pdf HTMap.pptx

51 Wednesday Closing Remarks

Speaker: Miron Livny (UW-Madison CHTC)

Wednesday Night Social

Thu, May 23

8:00 AM Coffee and Registration

Thursday AM Systems Administration Talks 1

Conveners: Greg Thain (Center for High Throughput Computing), Mr Zach Miller (UW Madison)

52 HTCondor Architecture

Speaker: Mr Todd Tannenbaum (University of Wisconsin)

Slides TannenbaumT_Architecture.pdf TannenbaumT_Architecture.pptx

53 Installation and configuration of a HTCondor Pool

Speaker: Tim Theisen (UW-Madison CHTC)

Slides 2019-Install-HTCondor.odp 2019-Install-HTCondor.pdf

54 Moving your job's data

Speaker: Brian Bockelman (Morgridge Institute for Research)

Slides Data_Access_with_HTCondor.pdf

55 Event-Sourced Monitoring of your HTCondor Cluster

Speaker: Mr Kevin Retzke (Fermilab)

Slides EventSourcedMonitoring-Retzke-HTCondorWeek2019.pdf EventSourcedMonitoring-Retzke-HTCondorWeek2019.pptx

10:40 AM Break

Thursday AM Systems Administration Talks 2

Conveners: Ian Ross (U. Wisconsin), Mr Jaime Frey (Center for High-Throughput Computing)

56 Federating HTCondor Pools

Speaker: Mr Greg Thain (Center for High Throughput Computing)

Slides Federating_Pools.pptx

57 SciTokens and Credential Management

Speakers: Jason Patton (University of Wisconsin), Mr Zach Miller (UW Madison)

Slides SciTokens-HTCondorWeek2019.pdf SciTokens-HTCondorWeek2019.pptx

58 Schedd Job Transforms

Speaker: Mr John Knoeller (University of Wisconsin, Madison)

Slides TJs_Schedd_Transforms_Condor_Week_2019.pptx

12:30 PM Lunch