Jim Blythe
ISI, USC

AI-based approaches to workflow management in Grids
---------------------------------------------------

Abstract

Grid computing promises the ability for users to harness the power of large
numbers of heterogeneous, distributed resources, including computing
resources, data storage systems and instruments. The vision is to enable
users and applications to seamlessly access these resources to solve complex
large-scale problems. Data intensive scientific applications are no longer
being developed as monolithic bodies of code. Instead, standalone
application components are combined to process the data in various ways. The
applications can now be viewed as complex workflows, consisting of various
transformations performed on the data. These workflows often require the
Grid to execute efficiently because of the large amounts of computation and
data involved.  In current environments, users need to discover resources
manually and schedule the jobs directly onto the Grid, essentially composing
detailed workflow descriptions by hand. This leaves users struggling with
the complexity of the Grid and weighing which resources to use, where to run
the computations, where to access the data etc.

We are working to automate this workflow generation process as far as
possible and to provide knowledge-based assistance to users where needed. We
describe an architecture that integrates several grid services to take a
high-level workflow specification, locate appropriate resources and oversee
its execution on the Grid. Within this architecture we use AI planning to
generate workflows from logical descriptions of the required data. The
system has been used to execute gravitational physics workflows involving
hundreds of individual tasks. Some of the challenges for planning in Grids
include (1) how to distribute the knowledge and the reasoning processes for
workflow generation across the Grid and (2) monitoring and repairing
workflows as the environment changes during their execution.

This is joint work with Yolanda Gil, Ewa Deelman and others at ISI.


Bio:

Jim Blythe is a computer scientist in the Information Sciences Institute at
USC. His interests include planning, knowledge acquisition and intelligent
user interfaces. He is currently working on planning for Grids and on
acquiring advice for planning systems from end users. He is chairing a
workshop this year on planning and scheduling for web and grid services at
the international planning conference, and is a manager of the Global Grid
Forum research group on Workflows. He received his PhD from Carnegie Mellon
University in 1998 on planning under uncertainty.