Workflows Community Summit: Advancing the State-of-the-art of Scientific Workflows Management Systems Research and Development

by   Rafael Ferreira da Silva, et al.

Scientific workflows are a cornerstone of modern scientific computing, and they have underpinned some of the most significant discoveries of the last decade. Many of these workflows have high computational, storage, and/or communication demands, and thus must execute on a wide range of large-scale platforms, from large clouds to upcoming exascale HPC platforms. Workflows will play a crucial role in the data-oriented and post-Moore's computing landscape as they democratize the application of cutting-edge research techniques, computationally intensive methods, and use of new computing platforms. As workflows continue to be adopted by scientific projects and user communities, they are becoming more complex. Workflows are increasingly composed of tasks that perform computations such as short machine learning inference, multi-node simulations, long-running machine learning model training, amongst others, and thus increasingly rely on heterogeneous architectures that include CPUs but also GPUs and accelerators. The workflow management system (WMS) technology landscape is currently segmented and presents significant barriers to entry due to the hundreds of seemingly comparable, yet incompatible, systems that exist. Another fundamental problem is that there are conflicting theoretical bases and abstractions for a WMS. Systems that use the same underlying abstractions can likely be translated between, which is not the case for systems that use different abstractions. More information:


page 1

page 7


Workflows Community Summit: Bringing the Scientific Workflows Community Together

Scientific workflows have been used almost universally across scientific...

Workflows Community Summit: Tightening the Integration between Computing Facilities and Scientific Workflows

The importance of workflows is highlighted by the fact that they have un...

Workflows Community Summit 2022: A Roadmap Revolution

Scientific workflows have become integral tools in broad scientific comp...

WfBench: Automated Generation of Scientific Workflow Benchmarks

The prevalence of scientific workflows with high computational demands c...

A Community Roadmap for Scientific Workflows Research and Development

The landscape of workflow systems for scientific applications is notorio...

From Piz Daint to the Stars: Simulation of Stellar Mergers using High-Level Abstractions

We study the simulation of stellar mergers, which requires complex simul...

ProvLet: A Provenance Management Service for Long Tail Microscopy Data

Provenance management must be present to enhance the overall security an...

Please sign up or login with your details

Forgot password? Click here to reset