Cooperative Computing Lab

CCTools 2.4.4 Released

Mon, 25 Aug 2008 11:15:50 EST

We are pleased to announce release 2.4.4 of the Cooperative Computing Tools, including Parrot, Chirp, and other tools which may be downloaded here:

http://www.cse.nd.edu/~ccl/software/download.shtml

A recent version of the patched Red Hat kernel installed by up2date inhibits access to the special file /proc/X/mem, which caused previous versions of Parrot to stop functioning with the error "Permission denied."

This release works around that bug in the kernel.

Troubleshooting at Grid 2008

Wed, 25 Jun 2008 15:43:53 EST

David Cieslak will present a paper titled Troubleshooting Thousands of Jobs on Production Computing Grids Using Data Mining Techniques at Grid 2008 in Japan. This work demonstrates techniques for drawing conclusions such as "Your jobs fail on Linux 2.4 with less than 16GB RAM" from complex workloads of thousands of jobs.

Datalab at HPDC 2008

Wed, 25 Jun 2008 15:40:05 EST

Brandon Rich presented a poster on DataLab: Active Storage for Data Drive Scientific Computing at High Performance Distributed Computing in Boston. DataLab is a system for robustly performing large data parallel workloads on hundreds of active storage nodes, using distributed transaction concepts to create a robust system.

CCTools 2.4.3 Released

Wed, 11 Jun 2008 21:05:54 EST

We are pleased to announce release 2.4.3 of the Cooperative Computing Tools, including Parrot, Chirp, and other tools which may be downloaded here.

Major items in this release:

New documentation for the Chirp APIs
Improvements to the Chirp server:
- New support for streaming I/O and server-side FIFOs
- New support for complex path names using RFC 2396 encoding.
- New P right that allows a user only to put new files.
Improvements to the Chirp clients:
- Strided I/O routines for array access.
- Client interfaces for streaming I/O.
- Improvement to the build flags that make it easier for other programs to compile against Chirp.
Miscellaneous bug fixes:
- Fixed case where server improperly returns "permission denied" instead of "file not found".es in the server.

NSF Summer REU Grant

Sun, 01 Jun 2008 21:11:52 EST

The CCL has received a grant from the National Science Foundation which will support two undergraduates in summer 2008 to participate in the construction of a novel repository for biometric data.

Researchers at Notre Dame have collected tens of thousands of such images and videos, and design new algorithms for identifying and matching people based upon these measurements. Answering these questions is very computation and data intensive. A large scale study of a new matching algorithm could take many CPU years to complete. To attack these problems in a reasonable amount of time, we must enlist hundreds of CPUs to work on different portions of the problem. While we have demonstrated the practicality of this idea with some custom programming, the overall system is not (yet) easy to use for end researchers. To solve this problem, the participants in this program will construct a well-organized repository of biometric data, connect it to our campus distributed computing system, and create an interface that makes it easy to specify and execute large biometric jobs.

CCL REU Program

Distributed Computing for Biometrics

CCL to Participate in Google/IBM Cluster Pilot

Tue, 20 May 2008 16:50:19 EST

In the 2008-2009 school year, junior and senior students in the CSE department will have the opportunity to learn techniques for large scale computing on clusters used by large internet service providers. Google and IBM have announced the 2008 Academic Cluster Initiative, which will provide a 1000-node machine for use by university students around the country. Students in Professor Douglas Thain's distributed systems and operating systems classes will learn how to write large data intensive programs in languages such as Map-Reduce on this cluster.

Papers at IPDPS 2008

Tue, 01 Apr 2008 20:24:24 EST

At IPDPS 2008 in Miami, Chris Moretti presented All-Pairs: An Abstraction for Data Intensive Cloud Computing, and Kyle Wheeler presented QThreads: An API for Programming with millions of Lightweight Threads.

Parrot Flies at Fermilab

Tue, 01 Jan 2008 11:15:05 EST

Parrot and the GROW filesystem are in production use at Fermi National Laboratory. The CDF experiment exploits the open Science Grid to run a large number of monte carlo simulations. Because the simulation code is highly complex and not practical to install at all sites, Parrot and the GROW filesystem are used to access the software on demand from Fermilab. More details here.