Speaker
Description
I use CHTC and OSPool to run a rainfall frequency analysis pipeline across 200+ watersheds as part of FEMA's National Flood Insurance Program work. DAGMan was essential for automating job submission and output processing at this scale, but managing pipeline failures proved challenging -- specifically, ensuring that every daily precipitation file was properly analyzed before downstream steps proceeded. A failure-and-retry approach led to redundant computation, while simply continuing risked carrying missing or corrupted data forward into later analyses. To address this, I developed submit files that intake dynamically updated file lists, whittling down as outputs are confirmed present. This prevents the pipeline from advancing with incomplete data while also avoiding redundant reprocessing of files that completed successfully. I could also focus instead more on the pipeline itself if that would be preferred. I am available to present in person on 6/10 or 6/11.