HTCondor Week 2019

May 20, 2019, 8:00 AM → May 23, 2019, 2:00 PM America/Chicago

Discovery Building on the University of Wisconsin-Madison Campus

Brian Bockelman (University of Nebraska-Lincoln), Mark Coatsworth (University of Wisconsin), Todd Tannenbaum (University of Wisconsin)

Monday, May 20

8:00 AM → 9:00 AM Coffee and Registration

9:00 AM → 10:20 AM Monday AM Advanced Tutorials 1

Conveners: Brian Lin (UW-Madison CHTC), Mr John Knoeller (University of Wisconsin, Madison)

9:00 AM Philosophy of HTCondor

Speaker: Mr Greg Thain (Center for High Throughput Computing)

Slides Philosophy.pptx

9:30 AM ClassAd Advanced Tutorial

Speaker: Mr Jaime Frey (Center for High-Throughput Computing)

Slides CW19-classads.ppt

10:00 AM HTCondor in the Cloud

Speaker: Mr Todd Miller (CHTC)

Slides condor_annex-Tutorial-v7.pptx

9:00 AM → 10:20 AM Monday AM Introductury Tutorials 1

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

9:00 AM Introduction to CHTC

Speaker: Lauren Michael (UW-Madison)

Slides 2019HTCondorWeek_IntroCHTC_LMichael.pdf 2019HTCondorWeek_IntroCHTC_LMichael.pptx

9:20 AM Intro to HTCondor Job Submission

Speaker: Christina Koch (UW Madison)

Slides 2019-Koch-User-Tutorial.pdf 2019-Koch-User-Tutorial.pptx

10:20 AM → 10:45 AM Break

10:45 AM → 12:15 PM Monday AM Advanced Tutorials 2

Conveners: Ian Ross (U. Wisconsin), Mr Zach Miller (UW Madison)

10:45 AM Negotiator Policy and Configuration

Speaker: Mr Greg Thain (Center for High Throughput Computing)

Slides Negotiator.pptx

11:55 AM Advanced Submission Tutorial

Speaker: Mr John Knoeller (University of Wisconsin, Madison)

Slides TJs_Advanced_Submit_Condor_Week_2019.pptx

10:45 AM → 12:15 PM Monday AM Introductory Tutorials 2

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

10:45 AM HTCondor Job Testing, Troubleshooting, and Special Use Cases

Speaker: Christina Koch (UW Madison)

11:15 AM Manage Workflows with HTCondor’s DAGMan

Speaker: Lauren Michael (UW-Madison)

Slides 2019HTCondorWeek_AutomatedWorkflowsDAGMan_LMichael.pdf 2019HTCondorWeek_AutomatedWorkflowsDAGMan_LMichael.pptx

12:15 PM → 1:15 PM Lunch

1:15 PM → 2:45 PM Monday PM Advanced Workshops 1

Conveners: Aaron Moate (UW-Madison CHTC), Greg Thain (Center for High Throughput Computing)

1:15 PM Python Bindings Tutorial

Speaker: Brian Bockelman (University of Nebraska-Lincoln)

Slides Bockelman-HTCondorWeek-Python-2019.pdf http://bit.ly/htcpy-stable

1:55 PM Work Queue tutorial

Speaker: Dr Benjamin Tovar (University of Notre Dame)

Slides 05-20-2019-Work_Queue-@-CondorWeek.pdf

1:15 PM → 2:45 PM Monday PM CHTC Research 1

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

1:15 PM Inferring Effective Connectivity From High-Dimensional ECoG Recordings

Speaker: Chris Endemann (UW-Madison Department of Anesthesiology)

Slides HTCondorWeek2019_Endemann_EffectiveConnectivity.pdf HTCondorWeek2019_Endemann_EffectiveConnectivity.pptx

1:35 PM Genomic ancestry analysis in wild hybrid house mice

Speaker: Megan Frayer (UW-Madison Laboratory of Genetics)

Slides Frayer_HTCondorWeek_2019_web.pdf Frayer_HTCondorWeek_2019_web.pptx

1:55 PM Uncertainty propagation in Fuel Cycle simulation using HTC

Speaker: Baptiste Mouginot (UW-Madison Department of Engineering)

Slides Fuel_Cycle_simulation_using_HTC_Condor.pdf

2:15 PM Alignment and CNV analysis in Cattle

Speaker: Beth Lett (UW-Madison Department of Animal Science)

Slides BL_Presentation.pptx

2:45 PM → 3:15 PM Break

2:45 PM → 3:15 PM Break

3:15 PM → 4:45 PM Monday PM Advanced Workshops 2

Conveners: Carl Edquist (UW-Madison CHTC), Mark Coatsworth (University of Wisconsin)

3:15 PM Pegasus Workshop

Speaker: Mr Karan Vahi (Pegasus - Team USC Information Sciences Institute)

Slides pegasus-chtc-tutorial-final.pdf

3:15 PM → 4:15 PM Monday PM CHTC Research 2

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

3:15 PM Image Analysis and Computation-Based Phenotyping

Speaker: Nathan Miller (UW-Madison Department of Botany)

3:35 PM Grid Search Methods for Modeling Polymerization Kinetics

Speaker: Katie Ziebarth (UW-Madison Department of Chemistry)

Slides HTCondorWeek2019_KZiebarth_Final.pdf HTCondorWeek2019_KZiebarth_Final.pptx

3:55 PM Studying RNA Virus Replication with Cryo-Electron Microscopy on HTC

Speaker: Hong Zhan (Morgridge Institute for Research)

Slides HTC_presentation_HZ.pdf HTC_presentation_HZ.pptx

4:15 PM → 5:45 PM Ice Cream Mixer: talk with other HTCondor users

Tuesday, May 21

8:00 AM → 8:55 AM Coffee and Registration

8:55 AM → 10:30 AM Tuesday AM Welcome and Keynote

Conveners: Mark Coatsworth (University of Wisconsin), Mr Todd Tannenbaum (University of Wisconsin)

8:55 AM Welcome to HTCondor Week

Speaker: Miron Livny (UW-Madison CHTC)

Slides 05-19-CW.pptx

9:00 AM Ice Fishing for Neutrinos: Exploring the mysteries of the cosmos from beneath the South Pole

Speaker: Dr Francis Halzen (UW-Madison WIPAC)

10:00 AM Computer and Data Sciences at UW-Madison

Speaker: Remzi Arpaci-Dusseau (UW-Madison Department of Computer Sciences)

Slides CondorHTC-Spring19_(3).pptx

10:15 AM → 10:45 AM Break

10:45 AM → 12:15 PM Tuesday AM Technical Talks

Conveners: Aaron Moate (UW-Madison CHTC), Ian Ross (U. Wisconsin)

10:45 AM IceCube Computing

Speaker: Benedikt Riedel (UW-Madison WIPAC)

Slides HTCondorWeek_2019.pdf HTCondorWeek_2019.pptx

11:10 AM HTCondor in Foxconn Hyperscale AI HPC Cloud

Speaker: Hank Du (Foxconn Industrial Internet)

Slides HTCondor_DCT_20190411_v4.pdf

11:25 AM Orpheus: Efficient Distributed Machine Learning via System and Algorithm Co- design

Speaker: Pengtao Xie (Petuum Inc.)

11:45 AM What's new in HTCondor? What is upcoming?

Speaker: Mr Todd Tannenbaum (University of Wisconsin)

Slides TannenbaumT_WhatsNew_HTCondor_Week_2019.pptx

12:15 PM → 1:15 PM Lunch

1:15 PM → 3:20 PM Tuesday PM Technical Talks 1

Conveners: Mr Jaime Frey (Center for High-Throughput Computing), Tim Slauson (UW-Madison CHTC)

1:15 PM Space Telescope Science Institute (STScI) and HTCondor – Reporting on today and exploring tomorrow.

Speaker: Mrs Mary Romelfanger (Space Telescope Science Institute)

Slides HTCondor_2019_STScI.pdf HTCondor_2019_STScI.pptx

1:40 PM Simplifying submitting for our users with a custom submission API

Speaker: Mr Collin Mehring (Dreamworks Animation)

Slides HTCondorWeek_2019.pptx

2:05 PM Updates from LIGO

Speaker: James Clark (Georgia Institute of Technology (LIGO))

Slides JamesClarkCondorWeek.pdf

2:30 PM 'Startd flocking' at and around the Milan Physics Dept.

Speaker: Francesco Prelz (INFN - Sezione di Milano)

Slides http://www.mi.infn.it/~prelz/condor_week_2019/

2:55 PM HTCondor in Astronomy at NCSA

Speakers: Greg Daues (NCSA), Michael Johnson (National Center for Supercomputing Applications)

Slides HTCondor_NCSA_2019.pdf HTCondor_NCSA_2019.pptx

3:20 PM → 3:45 PM Break

3:45 PM → 5:05 PM Tuesday PM Technical Talks 2

Conveners: Josh Karpel (UW-Madison CHTC), Matyas Selmeci (UW-Madison CHTC)

3:45 PM An opportunistic HTCondor pool inside a interactive-friendly Kubernetes cluster

Speaker: Mr Igor Sfiligoi (University of California San Diego)

Slides HTCondor_PRP_2019.pdf HTCondor_PRP_2019.pptx

4:10 PM Pegasus : Introducing Integrity to Scientific Workflows

Speaker: Mr Karan Vahi (Pegasus - Team USC Information Sciences Institute)

Slides vahi-pegasus-cw-2019-final.pdf vahi-pegasus-cw-2019-final.pptx

4:35 PM AI platform on HTcondor

Speaker: Mr Vishnu Vettrivel (Wisecube)

Slides ai_platform_htcondor_wisecube.pdf ai_platform_htcondor_wisecube.pptx

5:00 PM Tuesday Closing Remarks

Speaker: Miron Livny (UW-Madison CHTC)

5:05 PM → 7:00 PM Amazon Web Services Social

Wednesday, May 22

8:00 AM → 9:00 AM Coffee and Registration

9:00 AM → 10:35 AM Wednesday AM Campus Research

Conveners: Christina Koch (UW Madison), Lauren Michael (UW-Madison)

9:00 AM Facilitating High Throughput Computing

Speaker: Lauren Michael (UW-Madison)

Slides 2019HTCondorWeek_RCFacilitation_LMichael.pdf 2019HTCondorWeek_RCFacilitation_LMichael.pptx

9:20 AM Improving the quality of my research on urban residential hydrology by increasing the quantity of simulations using HTCondor

Speaker: Carolyn Voter (University of Wisconsin-Madison)

Slides 2019.05_htcondor_week_presentation_cvoter.pptx

9:45 AM Analyzing Soil Microbial Community Sequencing Data with the CHTC

Speaker: Thea Whitman (UW-Madison Department of Soil Science)

Slides CHTC_Whitman_2019.pdf

10:10 AM High Throughput Computing For the Generation of a Self-Stabilizing Laser Sail Capable of Exploring Beyond Our Solar System

Speaker: Joel Siegel (UW-Madison Department of Physics)

Slides PodHandler.pdf PodHandler.pptx

10:35 AM → 11:00 AM Break

11:00 AM → 12:15 PM Wednesday AM Technical Talks

Conveners: Tim Theisen (UW-Madison CHTC), Mr Todd Miller (CHTC)

11:00 AM High-throughput processing of telemetry data

Speakers: Mr Colton Mikes (Texas Tech University), Dr DJ Lee (Brigham Young University), Dr Richard Erickson (US Geological Survey)

Slides USGS_HTCondor-updated.pdf USGS_HTCondor-updated.pptx

11:25 AM Deploying and managing containers on OSG

Speaker: Marco Mambelli (Fermilab)

Slides GlideinWMS-condor_week2019v2.pdf

11:50 AM Scientific Computing using AWS and HTCondor

Speaker: Dr Sanjay Padhi (AWS)

Slides AWS_HTCondor2019_pub.pdf

12:15 PM → 1:15 PM Lunch

1:15 PM → 3:20 PM Wednesday PM Technical Talks 1

Conveners: Brian Lin (UW-Madison CHTC), Josh Karpel (UW-Madison CHTC)

1:15 PM HTCondor Solutions for the Future of the CMS Submission Infrastructure

Speaker: Dr JAMES LETTS (UCSD)

Slides JLETTS-CMSHTCondorWeek2019_.pdf

1:40 PM Managing a Dynamic Sharded Pool

Speaker: Anthony Tiradani (Fermilab National Accelerator Laboratory)

Slides FNAL_HTCondor_Week_2019.pdf FNAL_HTCondor_Week_2019.pptx

2:05 PM HTCondor with google Cloud Platform

Speaker: Michiru Kaneda (The University of Tokyo)

Slides 20190522_kaneda_htcondor_week.pdf 20190522_kaneda_htcondor_week.pptx

2:30 PM HTCondor Solutions for Several Scenarios at IHEP

Speaker: Jiaheng Zou (Institute of High Energy Physics (IHEP) of the Chinese Academy of Sciences (CAS))

Slides HTCondor_at_IHEP.pdf HTCondor_at_IHEP.pptx

2:55 PM Job Submission via Share Filesystem: Running HTCondor Jobs on Isolated HPC Machines

Speaker: Mr Jaime Frey (Center for High-Throughput Computing)

Slides CW19-FileSystemJobs.ppt

3:20 PM → 3:50 PM Break

3:50 PM → 5:10 PM Wednesday PM Technical Talks 2

Conveners: Jason Patton (University of Wisconsin), Tim Slauson (UW-Madison CHTC)

3:50 PM Python bindings based monitoring at CMS

Speaker: Diego Davila (CERN)

Slides python_bindings_monitoring.pdf

4:15 PM Thinking about HTC Notebooks: High Throughput Computing via Jupyter Notebooks

Speaker: Mr Todd Tannenbaum (University of Wisconsin)

Slides TannenbaumT_HTC_Notebooks.pdf TannenbaumT_HTC_Notebooks.pptx

4:30 PM HTMap: High-Throughput Computing in Python

Speaker: Mr Josh Karpel (University of Wisconsin - Madison)

Slides HTMap.pdf HTMap.pptx

4:50 PM Wednesday Closing Remarks

Speaker: Miron Livny (UW-Madison CHTC)

6:00 PM → 8:00 PM Wednesday Night Social

Thursday, May 23

8:00 AM → 9:00 AM Coffee and Registration

9:00 AM → 10:40 AM Thursday AM Systems Administration Talks 1

Conveners: Greg Thain (Center for High Throughput Computing), Mr Zach Miller (UW Madison)

9:00 AM HTCondor Architecture

Speaker: Mr Todd Tannenbaum (University of Wisconsin)

Slides TannenbaumT_Architecture.pdf TannenbaumT_Architecture.pptx

9:25 AM Installation and configuration of a HTCondor Pool

Speaker: Tim Theisen (UW-Madison CHTC)

Slides 2019-Install-HTCondor.odp 2019-Install-HTCondor.pdf

9:50 AM Moving your job's data

Speaker: Brian Bockelman (Morgridge Institute for Research)

Slides Data_Access_with_HTCondor.pdf

10:15 AM Event-Sourced Monitoring of your HTCondor Cluster

Speaker: Mr Kevin Retzke (Fermilab)

Slides EventSourcedMonitoring-Retzke-HTCondorWeek2019.pdf EventSourcedMonitoring-Retzke-HTCondorWeek2019.pptx

10:40 AM → 11:00 AM Break

11:00 AM → 12:30 PM Thursday AM Systems Administration Talks 2

Conveners: Ian Ross (U. Wisconsin), Mr Jaime Frey (Center for High-Throughput Computing)

11:00 AM Federating HTCondor Pools

Speaker: Mr Greg Thain (Center for High Throughput Computing)

Slides Federating_Pools.pptx

11:25 AM SciTokens and Credential Management

Speakers: Jason Patton (University of Wisconsin), Mr Zach Miller (UW Madison)

Slides SciTokens-HTCondorWeek2019.pdf SciTokens-HTCondorWeek2019.pptx

12:00 PM Schedd Job Transforms

Speaker: Mr John Knoeller (University of Wisconsin, Madison)

Slides TJs_Schedd_Transforms_Condor_Week_2019.pptx

12:30 PM → 1:30 PM Lunch