=======================================================
 MC#.METACLUSTER USER'S GUIDE
=======================================================

MC#.MetaCluster is the system which allows you execute your MC# programs
in distributed mode on several collaborating clusters (we call the colection
of such clusters as `metacluster').

Terminology:

Generally speaking, metacluster is the collection of collaborating clusters,
cluster is the collection of nodes and each node can have several processors.

If you only have one cluster please use MC#.Cluster system.
It is much easier to use it and currently it is much more stable system.
You can download it on our site:
http://u.pereslavl.ru/~vadim/MCSharp/

This package contains the compiler and Runtime System for the
MC#.MetaCluster programming system for Linux/Unix-based platforms
(mcsharp.dll library, Meta Resource Manager, Resource Manager and
Work Node services).
Also it contains some cluster management utilities and examples
of programs written in MC# programming language.

_______________________________________________________
SYSTEM REQUIREMENTS:

>= Mono 1.2.2.1
rsh or ssh

_______________________________________________________
OPERATING SYSTEMS:

Red Hat >= 8.0, Mandrake 9.1/x86, SuSE >= 8.2

_______________________________________________________
INSTALLATION:

Perform the following operations on all your clusters:

1. Untar the archive into the folder '~/mcsharp.meta' (or any other folder)

2. If you used MC#.Cluster system make sure that you have disabled environment
variables for MC#.Cluster in your ~/.bashrc and ~/.bashrc_profile files.
Currently MC#.Cluster and MC#.MetaCluster cannot operate under the same
system login simultaneously.

3. Compilation. It's supposed that mcs (Mono C# compiler) and
gcc (C++ compiler) had already been installed on your system.

cd ~/mcsharp.meta
make

4. Add the following environment variables:

$MCSGPATH    - the location of main directory of MC# is specified here
$MONO_PATH   - Mono will search for assemblies in these directories
$PATH        - add here the directory '~/mcsharp.meta/bin'

This can be done by adding the following lines into files '~/.bashrc'
and '~/.bashrc_profile':

export MCSGPATH=/home/yourhomedir/mcsharp.meta
export MONO_PATH=$MCSGPATH/bin:$MONO_PATH
export PATH=$MCSGPATH/bin:$PATH

Please note, that these environment variables must be added into both files!
Also make sure that $HOME variable is set up properly on your systems and
don't forget to restart your shell after you change these variables!

Note: at the end of installation installer will display the environment
variables which must be added to your bashrc files.

5. Make sure that changing nodes via rsh/ssh doesn't require
entering any passwords. If it is asking to enter passwords then rsh/ssh
must be configured properly. Read the following article to get more information:
http://u.pereslavl.ru/~vadim/MCSharp/docs/openssh/OpenSSH_Public_Key_Authentication.php

6. Make sure that firewalls are disabled or configured properly.
MC#.MetaCluster Runtime System relies heavily on using TCP/IP ports for
communications. It is required that it should be possible to use ports
starting from 10000 till 50000 on all nodes of the metacluster.

_______________________________________________________
COMPILATION OF PROGRAMS WRITTEN IN MC#

In this package there's a compiler called 'mcsc' for
compilation of MC# programs. There're two stages of compilation:
- MC#-code is translated into C# code
- C# code is compiled by 'mcs' compiler (from Mono platform).

Note:
 all parameters passed to 'mcsc' compiler will be transferred to C# compiler (mcs)

Usage examples:
 mcsc fib.mcs
 mcsc fib.mcs /out:myfib.exe
 mcsc /t:library fib.mcs

You can compile MC# programs into DLLs
 mcsc a.mcs /t:library /out:a.dll

Then you can include it into the program
 mcsc /r:a.dll b.mcs

You can mix MC# and C# files
 mcsc a.mcs b.cs

Or compile several MC# files at a time
 mcsc a.mcs b.mcs

If you need only to translate MC# files into C# code you can
use option `/translateonly'.

_______________________________________________________
UNDERSTANDING THE RUNTIME SYSTEM:

Programs, compiled by MC# compiler can be executed either autonomously
in local mode or in distributed mode on the metacluster.
In the second case you should specify the configuration file of the metacluster
with the help of option '/cluster' (or '-cluster') followed by the location of
metacluster configuration file. MetaCluster configuration file is a quite simple
XML-file with the following structure:

----
<metacluster id="skif and zoot" host="skif" port="10000">
 <cluster id="skif" host="node-11" port="20003" accesstype="rsh"
   mcsharppath="/home/vadim/mcsharp.meta/">
  <node host="node-11" port="20002" processors="2" />
  <node host="node-12" port="20002" processors="2" />
  <node host="node-13" port="20002" processors="2" />
  <node host="node-14" port="20002" processors="2" />
  <node host="node-21" port="20002" processors="2" />
  <node host="node-22" port="20002" processors="2" />
  <node host="node-23" port="20002" processors="2" />
  <node host="node-24" port="20002" processors="2" />
  <node host="node-31" port="20002" processors="2" />
  <node host="node-32" port="20002" processors="2" />
  <node host="node-33" port="20002" processors="2" />
  <node host="node-34" port="20002" processors="2" />
  <node host="node-41" port="20002" processors="2" />
  <node host="node-42" port="20002" processors="2" />
  <node host="node-44" port="20002" processors="2" />
 </cluster>
 <cluster id="zoot" host="node-1-1" port="20003" accesstype="rsh"
   mcsharppath="/home/vadim/mcsharp.meta/">
  <node host="node-1-1" port="20002" processors="2" accesstype="ssh" />
  <node host="node-1-2" port="20002" processors="2" accesstype="ssh" />
  <node host="node-2-1" port="20002" processors="2" accesstype="ssh" />
  <node host="node-2-2" port="20002" processors="2" accesstype="ssh" />
 </cluster>
</metacluster>
----

As you can see at the top level there is one metacluster element which describes
the properties of Meta Resource Manager. It is responsible for managing the load
of the clusters as well as for organizing communications between frontends of
the clusters and metacluster' frontend.

Meta Resource Manager can communicate only with Resource Managers launched
on the frontends of the clusters or with Client programs launched on
metacluster's frontend.

Resource Managers can communicate only with their Working Nodes, copies of client
programs launched on current clusters and with Meta Resource Manager. The general
structure of network organization can be demonstrated by the following picture:

               Client program (Fib.exe)              -.
                          |                           | MetaCluster's frontend
           Meta Resource Manager (metarm.exe)        _|
              /                         \
   Resource Manager #1            Resource Manager #n (resman.exe) - Cluster's
     /              \               /              \                 frontend
Work Node #1      Work Node #k  Work Node #1       Work Node #l (worknode.exe)
    |                 |             |                  |
  Fib.exe           Fib.exe       Fib.exe            Fib.exe

This version of runtime utilizes 'rsh', 'ssh' or possibly other types of
rsh-compatible remote shells for metacluster components initialization.
You can specify the remote shell command in 'accesstype' attribute. This
attribute can be configured at the level of 'cluster' element or at
the level of node 'element'.

Note: rsh/ssh must be configured for transparrent logging from metacluster's
frontend to frontends of the clusters as well as from frontend of clusters
to working nodes of those clusters.

There's a 'gboot' utility in this package for cluster initialization.
File '$MCSGPATH/bin/nodes' will be loaded when configuration file is not
specified as a parameter.

Usage example:
 gboot /cluster /path/to/metacluster/configuration/file
or
 gboot

For example:

[vadim@skif bin]$ gboot
Initializing Meta Cluster......(OK)
_____________________________________________________________________________

Meta Cluster `skif and zoot' - skif:10000 (19 nodes, 38 processors) (OK)
 Cluster `skif' - node-11:20003 (OK)
 Cluster `zoot' - node-1-1:20003 (OK)
_____________________________________________________________________________

By default, user's programs are executing in local mode, i.e. as usual
mono applications. In that case it isn't necessary to launch the Meta Resource
Manager, Resource Managers and Work Nodes. It can be much easier to
write and debug applications in local mode, before you launch them on
production servers.

You can always get the information about the currently used metacluster
configuration by running the 'gnodes' command:

[vadim@skif bin]$ gnodes
_____________________________________________________________________________

The following configuration is currently being in use:

Meta Cluster `skif and zoot' - skif:10000 (19 nodes, 38 processors)
 Cluster `skif' - node-11:20003
  Node node-11[2]:20002
  Node node-12[2]:20002
  Node node-13[2]:20002
  Node node-14[2]:20002
  Node node-21[2]:20002
  Node node-22[2]:20002
  Node node-23[2]:20002
  Node node-24[2]:20002
  Node node-31[2]:20002
  Node node-32[2]:20002
  Node node-33[2]:20002
  Node node-34[2]:20002
  Node node-41[2]:20002
  Node node-42[2]:20002
  Node node-44[2]:20002
 Cluster `zoot' - node-1-1:20003
  Node node-1-1[2]:20002
  Node node-1-2[2]:20002
  Node node-2-1[2]:20002
  Node node-2-2[2]:20002
_____________________________________________________________________________

When you finish working with the cluster, you should finalize the Runtime on
your clusters. Command 'ghalt' is dedicated for this task.

To execute some command on all nodes of your cluster use 'do-all' command.
This command can operate only when the metacluster has been initialized by
'gboot' command.

For example:
 do-all killall -9 mono

This command will kill all mono applications on your metacluster.

There's a special built-in command for the same task called 'killmono' which
kills all running processes on all nodes of the metacluster, including nodes
where Meta Resource Manager and Resource Managers are running.

Sometimes this can really help to clear the non-responding processes on your
clusters when something goes wrong...

_______________________________________________________
STARTING USER`S COMPILED PROGRAMS

Local mode:
 mono fib.exe 35

This will start program in local mode, i.e. movable methods will be executed
as new threads. No network operations will occur.

Distributed mode:
 mono fib.exe 35 /cluster /path/to/metacluster/configuration/file

If you want to enable logging information you should use option '/withlog'.
This will generate separate log files for each node in folders
'~/.mcsharp/logs/' of cluster frontends.

Also, if you want to see more detailed information you can use option '/withdebug'.

This will start program in distributed mode on specified configuration.
In this case movable methods will be transferred across the network.

By default runtime system supposes that executable files are available
in the same directory on all nodes of the metacluster. However, sometimes it
is just not possible, due to different cluster configurations. Also it is
required somehow to synchronize files which are located on different clusters.

For this purposes there is a special utility 'apploader' which allows
to copy files from the specified directory to clusters and modify configuration
file to reflect the directory binding information.

[vadim@skif fib]$ apploader /?

This utility copies the content of specified directory to the clusters
and generates configuration file.
Usage:
 apploader [[[/src/directory/ /dst/directory/] config.in] config.out]
where
 - '/src/directory/' - from which directory files must be copied, the
   default is the current directory
 - '/dst/directory/' - into which directory files must be copied, the
   default is the current directory
 - 'config.in'  - name of the source file where the structure of metacluster
   is specified, the default is `nodes.in'
 - 'config.out' - name of the destination file where the modified structure of
   metacluster will be written, the default is `nodes.out'

For example:


[vadim@skif fib]$ apploader /home/vadim/mcsharp.meta/examples/fib
/home/vadim/mcsharp.meta/examples/fib $MCSGPATH/bin/nodes /home/vadim/nodes.out
Creating directory `/home/vadim/mcsharp.meta/examples/fib' on clusters...
 Cluster `skif' - node-11:20003 - (OK) - Directory is already existing
 Cluster `zoot' - node-1-1:20003 - (OK)
Copying file `/home/vadim/mcsharp.meta/examples/fib/Fib.cs' to...
 Cluster `skif' - node-11:20003 - (OK) - File is up-to-date
 Cluster `zoot' - node-1-1:20003 - (OK)
Copying file `/home/vadim/mcsharp.meta/examples/fib/Fib.mcs' to...
 Cluster `skif' - node-11:20003 - (OK) - File is up-to-date
 Cluster `zoot' - node-1-1:20003 - (OK)
Copying file `/home/vadim/mcsharp.meta/examples/fib/Fib2.cs' to...
 Cluster `skif' - node-11:20003 - (OK) - File is up-to-date
 Cluster `zoot' - node-1-1:20003 - (OK)
Copying file `/home/vadim/mcsharp.meta/examples/fib/Fib2.mcs' to...
 Cluster `skif' - node-11:20003 - (OK) - File is up-to-date
 Cluster `zoot' - node-1-1:20003 - (OK)
Copying file `/home/vadim/mcsharp.meta/examples/fib/Makefile' to...
 Cluster `skif' - node-11:20003 - (OK) - File is up-to-date
 Cluster `zoot' - node-1-1:20003 - (OK)
Copying file `/home/vadim/mcsharp.meta/examples/fib/fib.exe' to...
 Cluster `skif' - node-11:20003 - (OK) - File is up-to-date
 Cluster `zoot' - node-1-1:20003 - (OK)
Copying file `/home/vadim/mcsharp.meta/examples/fib/fib2.exe' to...
 Cluster `skif' - node-11:20003 - (OK) - File is up-to-date
 Cluster `zoot' - node-1-1:20003 - (OK)

[vadim@skif fib]$ cat ~/nodes.out
<metacluster id="skif and zoot" host="skif" port="10000">
  <cluster id="skif" host="node-11" port="20003" accesstype="rsh"
     mcsharppath="/home/vadim/mcsharp.meta/"
     workingdirectory="/home/vadim/mcsharp.meta/examples/fib">
    <node host="node-11" port="20002" processors="2" />
    <node host="node-12" port="20002" processors="2" />
    <node host="node-13" port="20002" processors="2" />
    <node host="node-14" port="20002" processors="2" />
    <node host="node-21" port="20002" processors="2" />
    <node host="node-22" port="20002" processors="2" />
    <node host="node-23" port="20002" processors="2" />
    <node host="node-24" port="20002" processors="2" />
    <node host="node-31" port="20002" processors="2" />
    <node host="node-32" port="20002" processors="2" />
    <node host="node-33" port="20002" processors="2" />
    <node host="node-34" port="20002" processors="2" />
    <node host="node-41" port="20002" processors="2" />
    <node host="node-42" port="20002" processors="2" />
    <node host="node-44" port="20002" processors="2" />
  </cluster>
  <cluster id="zoot" host="node-1-1" port="20003" accesstype="rsh"
     mcsharppath="/home/vadim/mcsharp.meta/"
     workingdirectory="/home/vadim/mcsharp.meta/examples/fib">
    <node host="node-1-1" port="20002" processors="2" accesstype="ssh" />
    <node host="node-1-2" port="20002" processors="2" accesstype="ssh" />
    <node host="node-2-1" port="20002" processors="2" accesstype="ssh" />
    <node host="node-2-2" port="20002" processors="2" accesstype="ssh" />
  </cluster>
</metacluster>


As you can see, apploader utility is adding 'workingdirectory' attributes
to 'cluster' elements. Now you can run your programs with this modified
configuration file:

[vadim@skif fib]$ mono fib.exe 35 /cluster ~/nodes.out /withlog
For n = 35 value is 14930352
_____________________________________________________________________________
--MC# Statistics-------------------------------------------------------------
Session initialization time: 00:00:02.5167440 / 2.516744 sec. / 2516.744 msec.
                 Total time: 00:00:03.1748650 / 3.174865 sec. / 3174.865 msec.
_____________________________________________________________________________

Your comments and suggestions are greatly appreciated!
Please, let us know what do you think about MC# project.
Visit http://u.pereslavl.ru/~vadim/MCSharp/

Or contact directly with:

 Yury P. Serdyuk                       Vadim B. Guzev
 Yury@serdyuk.botik.ru                 vadim@u.pereslavl.ru
 ICQ: #330402764                       ICQ: #62951762
 Responsible for Compilers             Responsible for Runtimes



Take care and don't program too much... Life's too short! :)
Developers of MC# Programming Language