Cyberaide Shell

Cyberaide Shell

Gregor von Laszewski, Andrew Younge, Leor Dilmanian, Xi He, Paresh Khatri, Sanket Patel, Fugang Wang

Cyberaide Shell is an advanced but simple to use system shell which facilitates the use of current and future cyberinfrastructure. It is based on the integration of semantically enhanced commands and a dynamic interfaces that allow users to access complex cyberinfrastructure in an easy, intuitive, and ad-hoc fashion. This is accomplished by abstracting the complexities of resource, task, workflow, and application management through a semantic command line interpreter. Through a service integration mechanism, the shell's functionality is exposed to a wide variety of frameworks and programming languages. This includes client interfaces for Java, Ruby, Python, Matlab, R, JavaScript.

The Goal of this project is fourfold:

  • we are developing a Grid and Cloud “Mediator” to which clients can connect in an ad-hoc fashion and interact with Grids and clouds. This Mediator uses standard WS-* technologies.
  • we are developing a “cybershell”/“gridshell” that provides a convenient command line interface to advanced cyberinfrastructure including Grids and Clouds.
  • we are providing a number of convenient client APIs and examples that allows users to easily interact with Grids and Clouds.
  • we are developing methodologies and tools that allows users to set up their own Grid and Cloud in an ad-hoc fashion with little effort.

Latest Gridshell Paper draft: vonLaszewski-08-gridshell.pdf

Example Job Mapping

The following example shows how to login to a teragrid resource without needing to remember all the details about the hostname. The keyword “ranger” is associated with the hostname and a lookup is performed on keywords in the shell that than get expanded based on the semantic context. Thus a login would be expanded to the hostname.

cyberaide> list resources
cyberaide> login ranger
...
ranger> exit
cyberaide> exec ranger /bin/uname

The following example shows how to set up a pool of resources and execute commands on it based on a scheduler.

cyberaide> set pool
cyberaide> add pool -name ranger ranger.tacc.teragrid.org
cyberaide> add pool -name cool cool.rit.edu
...
cyberaide> set -policy scheduler random pool  
...
cyberaide> set jobs
cyberaide> add -name job1 /home/gregor/bin/climate -512
cyberaide> add -name job2 /home/gregor/bin/climate -256
...
cyberaide> list jobs

No. Name  Command                        Resource  Status
==========================================================
1   job1  /home/gregor/bin/climate -512  none      Defined
2   job2  /home/gregor/bin/climate -256  none      Defined
  
cyberaide> machine = getNext pool
  
cyberaide> list resources

No. Nanme  Endpoint
==========================================================
1   ranger ranger.tacc.rit.edu 
2   cool   cool.rit.edu  

cyberaide> job = getNext job

cyberaide> exec machine job

Example Runtime Scheduling

While accessing a variety of information services, it is possible to schedule the jobs based on runtime information. We have combined a number of information services into an aide service that aides the user to use resources that have higher likely hood to get things accomplished. This includes the availability of resources, their utilization, and the availability of information provided by the application user to run the job.
cyberaide> shell
cyberaide> best_resource = aide -deadline 01/01/2009 03:00pm -nodes 5
cyberaide> submit -resource best_resource job

External References