HTCondor Week 2021

May 24, 2021, 11:00 AM → May 27, 2021, 2:45 PM America/Chicago

Virtual

Talk slides and recordings

Monday, May 24

11:00 AM → 12:30 PM HTC Big Picture

11:00 AM Introduction and Guidelines

Speaker: Mark Coatsworth (University of Wisconsin)

Slides HTCondor-Week-2021-Introduction-Guidelines.pdf HTCondor-Week-2021-Introduction-Guidelines.pptx

11:05 AM Plenary: Accelerating SARS-CoV-2 variant sequencing with HTCondor

Speaker: Dave O'Connor (UW Medical Foundation Professor of Pathology and Laboratory Medicine and UW-Madison)

Video Recording Plenary: Accelerating SARS-CoV-2 variant sequencing with HTCondor

11:30 AM Welcome to HTCondor Week 2021

Speaker: Miron Livny (UW-Madison CHTC)

Slides 05 21 CW.pdf 05 21 CW.pptx

Video Recording Welcome to HTCondor Week 2021

11:50 AM What's New? What's Improved?

Speaker: Todd Tannenbaum (University of Wisconsin)

Slides WhatsNew_HTCondorWeek_May_2021.pdf WhatsNew_HTCondorWeek_May_2021.pptx

Video Recording What's new? What's Improved?

12:10 PM HTC Philosophy

Speaker: Greg Thain (UW-Madison CHTC)

Slides HTPhilosophy.pptx

Video Recording HTC Philosophy

1:00 PM → 2:40 PM HTCondor 9.0

1:00 PM Introducing the HTCondor 9.0 Series (Users)

Speaker: Christina Koch (UW Madison)

Slides 2021-new-user-features.pdf 2021-new-user-features.pptx

Video Recording Introducing the HTCondor 9.0 Series (Users)

1:15 PM Introducing the HTCondor 9.0 Series (Admins)

Speaker: Greg Thain (Center for High Throughput Computing)

Slides NewAdmin.pptx

Video Recording Introducing the HTCondor 9.0 Series (Admins)

1:40 PM Security in HTCondor 9.0

Speaker: Brian Bockelman (Morgridge Institute for Research)

Slides HTCondor-Security-2021.pdf

Video Recording Security in HTCondor 9.0

2:00 PM Town Hall Discussion: Authorization and Identity

Speakers: Miron Livny (UW-Madison CHTC), Brian Bockelman (Morgridge Institute for Research), Jim Basney (University of Illinois Urbana-Champagne), Frank Würthwein (UCSD / Open Science Grid), Jeny Teheran (FermiLab)

Tuesday, May 25

11:00 AM → 12:30 PM Campus Research and Facilitation

11:00 AM Campus Research and Facilitation

Speaker: Lauren Michael (UW-Madison CHTC)

Slides 2021HTCondorWeek_CampusFacilitation_LMichael.pdf

Video Recording Campus Research and Facilitation

11:25 AM HTCondor Tutorials on Youtube

Speaker: Christina Koch (UW-Madison CHTC)

Slides 2021-youtube-talk.pdf 2021-youtube-talk.pptx

Video Recording HTCondor Tutorials on Youtube

11:40 AM Running COPASI biochemical simulations with HTCondor

COPASI is a widely used simulator for chemical and
biochemical reaction networks based on ordinary differential equations
or stochastic methods. It includes various analysis methods such as
optimization, parameter estimation, sensitivity analysis, and several
others. While COPASI is mostly used in a standalone GUI-based mode,
several compute-intesive tasks benefit from parallelization. We created
a web-based system which facilitates transforming such tasks into
smaller sub-tasks that can be run independently. This system then allows
the user to submit these tasks to HTCondor from the web interface, and
assembles the numerical results in their expected order. Thus the end
user never has to interact directly with HTCondor.

Speaker: Pedro Mendes (UConn Health)

Slides COPASI_with_HTCondor_2021.pdf

Video Recording Running COPASI biochemical simulations with HTCondor

12:05 PM Advancing data intensive science at George Washington University

Speaker: Clark Gaylord (George Washington University)

Slides 20210525-AdvancingDataIntensiveScience-HTCWeek.pdf

Video Recording Advancing data intensive science at George Washington University

1:00 PM → 2:40 PM Collaborator Presentations

1:00 PM High-throughput horizon screening for invasive species at USGS

The US Geological Survey (USGS) is currently leading a horizon scanning for new invasive species for the United States (US). This horizon scan is using a climate match to assess how climate in potential invasive species’ non-US range matches the climate in different parts of the US. We developed a high-throughput assessment using HTCondor to examine 8,000+ species. We will describe our workflow and how we created an R package, used Docker, and HTCondor for the assessment and provide suggestions for other people wanting to use R with HTCondor.

Speaker: Richard Erickson (USGS)

1:20 PM Building better tools with the help of the Open Science Grid

Speaker: Nick Cooley (University of Pittsburg)

Slides npc_OSG_v03.pdf

Video Recording Building better tools with the help of the Open Science Grid

1:40 PM Evolution of the CMS Submission Infrastructure to Support Heterogeneous Resources

The landscape of computing power available for the CMS experiment is already evolving from almost exclusively x86 processors, predominantly deployed at WLCG sites, towards a more diverse mixture of Grid, HPC and Cloud facilities, incorporating a higher fraction of non-CPU components, such as GPUs. The CMS Global Pool is consequently adapting to the heterogeneous resource scenario, aiming at making the new resource types available to CMS. An optimal level of granularity in their description and matchmaking strategy will be essential in order to ensure efficient allocation and matchmaking to CMS workflows. Current uncertainties involve what types of resources will be available in the future, how to prioritize diverse workflows to those diverse types, and how to deal with a diversity of policy preferences by the resource providers. This contribution will describe the present capabilities of the CMS Submission Infrastructure and its critical dependencies on the underlying tools (such as HTCondor and GlideinWMS), along with its necessary evolution towards a full integration and support of heterogeneous resources according to the CMS needs.

Speaker: Marco Mascheroni (CERN)

Slides Evolution of the CMS Submission Infrastructure to Support Heterogeneous Resources.pdf

Video Recording Evolution of the CMS Submission Infrastructure to Support Heterogeneous Resources

2:00 PM LIGO Monitoring and the Grafana Dashboard

Speaker: Michael Thomas (LIGO)

Slides HTCondor Week 2021.pdf

Video Recording LIGO Monitoring and the Grafana Dashboard

2:20 PM HTCondor in a Digitization Workflow : Helping Preserve Cultural Heritage

Digitization is an important aspect of the preservation and promotion of heritage materials. Once physical documents are too fragile or damaged to manipulate, the digital copy often becomes the only version that is available to the public. The digitization workflow must produce files that reliably meet high standards.

The combination of cycle scavenging and distributed computing of HTCondor allows the digital collections team to complete tasks faster with a small pool of 50 available workstations. The team submits projects to HTCondor through a web server that automatically prepares the submit file and input list.

Each task launches a Java application that handles file verification and executes tools such as Tesseract (optical character recognition), FFmpeg (audio, video file conversion) or ImageMagick (image conversion). Once the project is complete, the web server prepares a report using custom exit codes and informs the owner.

After being processed through HTCondor, the files are ready to be preserved for future generations. The projects for which the institution has dissemination rights then become available through our web platform : https://numerique.banq.qc.ca/patrimoine/

Speaker: David Lamarche (Bibliothèque et Archives nationales du Québec)

Slides BAnQ-HTCondorWeek2021.pdf BAnQ-HTCondorWeek2021.pptx

Video Recording HTCondor in a Digitization Workflow : Helping Preserve Cultural Heritage

Wednesday, May 26

11:00 AM → 12:30 PM HTCondor Technologies

11:00 AM Upgrading to HTCondor 9.0

Speaker: Todd Miller (CHTC)

Slides Getting and Upgrading HTCondor.pdf Getting and Upgrading HTCondor.pptx

Video Recording Upgrading to HTCondor 9.0

11:25 AM Managing Dropbox Workflows with HTCondor

Dropbox-driven workflows, where the appearance of new files in a given directory triggers work to be done on those inputs, are common in many contexts. Customarily these are implemented with cron jobs, or a service daemon in the system. The HTCondor platform has a number of features, such as built-in e-mail notifications, “crondor” for repeating jobs, and a well-conceived model of jobs and resources, which make building Dropbox workflows easier, and the result far more manageable.

The techniques I will describe were developed during 2020 to support an automated AI-driven visual and x-ray inspection process for silicon wafer and other component production which delivered $50 million worth of benefits, by reducing SME work and improving product quality and manufacturing yield as the data gathered was fed back into the component design, and was recognized in a prestigious Raytheon Missiles & Defense CIO Award.

Speaker: Michael Pelletier (Raytheon)

Video Recording Managing Dropbox Workflows with HTCondor

11:45 AM Python Workflows and htcondor.dags

Speaker: Patrick Godwin (LIGO)

Video Recording Python Workflows and htcondor.dags

12:05 PM Pegasus 5.0 + Ensemble Manager

Pegasus 5.0 is the latest stable release of Pegasus that was released in November 2020. A key highlight of this release, is a brand new Python3 based Pegasus API that allows users to compose workflows and to control their execution programmatically. This talk will give an overview of the new API and highlight various key improvements introduced that address system usability (including a comprehensive, yet easy-to-navigate documentation, and training), and the development of core functionalities for improving the management and processing of large, distributed data sets, and the management of experiment campaigns defined as ensembles.

Speaker: Karan Vahi (Pegasus Team - USC )

Slides vahi-cw2021-pegasus5.pdf

Video Recording Pegasus 5.0 + Ensemble Manager

1:00 PM → 3:00 PM Panel / Office Hours

1:00 PM Office Hours

2:00 PM Pegasus Tutorial

Speaker: Karan Vahi (Pegasus - Team USC Information Sciences Institute)

Slides vahi-cw2021-pegasus-tutorial-updated.pdf

2:00 PM Town Hall Discussion: Multiple GPU Jobs

Speakers: David Schultz (UW-Madison WIPAC), John Knoeller (University of Wisconsin, Madison), Josh Willis (LIGO), Todd Miller (UW-Madison CHTC)

Thursday, May 27

11:00 AM → 12:30 PM dHTC Technologies

11:00 AM dHTC for LHAASO Experiments

Speaker: Jingyan Shi (IHEP)

Slides dHTC+for+LHAASO_v2.2.pdf dHTC+for+LHAASO_v2.2.pptx

Video Recording Global pool with hand-rolled glidein

11:20 AM Unchaining JupyterHub: Running notebooks on resources without inbound connectivity

JupyterLab has become an increasingly popular platform for rapid prototyping, teaching algorithms or sharing small analyses in a self-documenting manner.

However, it is commonly operated using dedicated cloud-like infrastructures (e.g. Kubernetes) which often need to be maintained in addition to existing HTC systems. Furthermore, federation of resources or opportunistic usage are not possible due to a requirement of direct inbound connectivity to the execute nodes.

This talk presents a new, open development in the context of the JupyterHub batchspawner:
Extending the existing functionality to leverage the connection broker of the HTCondor batch system, the requirement for inbound connectivity to the execute nodes can be dropped, and only outbound connectivity to the Hub is needed.

Combined with a container runtime leveraging user namespaces, unprivileged CVMFS and the HTCondor file transfer mechanism, notebooks can not only be executed directly on existing local HTC systems, but also on opportunistically usable resources such as HPC centres or clouds via an overlay batch system.

The presented prototype paves the way towards a federation of heterogeneous and distributed resources behind a single point of entry.

Speaker: Oliver Freyermuth (University of Bonn)

Slides JupyterHub_Bonn.pdf

Video Recording Unchaining JupyterHub: Running notebooks on resources without inbound connectivity

11:45 AM Improving Kubernetes support for batch scheduling of high throughput and parallel jobs

Kubernetes is an open source cluster orchestration system whose popularity stems in part because it acts as a standard resource management interface across cloud providers and on-premises data centers. There is significant interest in managing HTCondor services and scheduling user jobs in Kubernetes clusters. These solutions often rely on running standard HTCondor daemons inside a container or developing custom Kubernetes operators to bridge the two services. Originally designed by Google, it remains a major contributor to Kubernetes which is now governed by the Cloud Native Computing Foundation. We will describe recent (1.21) and planned (1.22+) contributions to improve direct support for batch scheduling of high throughput and parallel jobs as well as developments in our Google Kubernetes Engine product, which offers Kubernetes clusters with reduced management overhead.

Speaker: Abdullah Gharaibeh (Google Cloud)

Slides Improving Batch Support in OSS Kubernetes.pdf

Video Recording Improving Kubernetes support for batch scheduling of high throughput and parallel jobs

12:05 PM HTCondor CE 5 and Job Router Transform Language

Speakers: Brian Lin (UW-Madison CHTC), John (TJ) Knoeller (UW-Madison CHTC)

Slides 2021-05-27.htcondor-ce-5-jr-transforms.pdf 2021-05-27.htcondor-ce-5-jr-transforms.pptx

Video Recording HTCondor CE 5 and Job Router Transform Language

1:15 PM → 2:45 PM CHTC Research

1:15 PM ML, Image Analysis for Livestock Data

Speaker: Joao Dorea (UW-Madison Animal & Dairy Sciences)

Slides htc_day2021_Doreatalk.pdf

Video Recording ML, Image Analysis for Livestock Data

1:35 PM Scaling Virtual Screening to Ultra-Large Virtual Chemical Libraries

Progress in chemical synthesis strategies has given rise to vast “make-on-demand” chemical libraries. Such libraries, now virtual, are bounded only by synthetic feasibility and are growing exponentially. Making and testing significant portions of such libraries on a new drug target is not feasible. We increasingly rely on computational approaches called virtual screening methods to help us navigate large chemical spaces and to prioritize the most promising molecules for testing. The main challenge now is to scale existing virtual screening methods, or develop new ones, with sufficient molecule throughput and scoring accuracy to accommodate ultra-large compound libraries. Here I will describe some promising approaches that leverage high-throughput computing to meet this challenge.

Speaker: Spencer Ericksen (UW-Carbone Cancer Center, Drug Development Core, Small Molecule Screening Facility)

Slides HTCondorWeek_Ericksen_2021-05-27_v1_no_extras.pdf

Video Recording Scaling Virtual Screening to Ultra-Large Virtual Chemical Libraries

1:55 PM Using high-throughput computing to develop precision mental health algorithms

Speaker: Gaylen Fronk (UW-Madison Addiction Research Center)

Slides fronk_htcondor_week_2021.pdf fronk_htcondor_week_2021.pptx

Video Recording Using high-throughput computing to develop precision mental health algorithms

2:15 PM IceCube Glideins and Autonomous Pilots

Speaker: Benedikt Riedel (UW-Madison WIPAC)

Slides HTCondor_Week_2021.pdf

Video Recording IceCube Glideins and Autonomous Pilots

2:35 PM Closing Remarks

Speaker: Miron Livny (UW-Madison CHTC)