Setting Up Experiments: setupexp

Setupexp was written to automatically run network experiments in the DiRT lab. You'll probably want to run setupnet before launching a set of experiments.

Comments from the setupexp source (/usr/dirt/bin/setupexp):

#
# Author: Mark Parris
# Created: Sometime before Feb. 14, 1999 
#
# Change Log:
#

# This program reads in an experiment script and generates a set of
# scripts in the specified directory which will run the commands at
# the the specified times on the specified machines.

# 
# Syntax notes:
#
# Note that this is a perl script that proesses other files which are 
# just in an "experiment scripting" language that I invented.  As such
# the syntax is NOT like PERL.   %{} is NOT an associative array.  
# @include is not an array named include.  Instead %{} is how I delimit
# a reference to a variable so %{bob} is a reference to the variable
# named "bob".  @include is just the keyword that indicates that 
# the file specified should be included here.  %{time}, $host, and %{DIR} 
# also have special uses described below as does "TIME="


#######################################################################
# VARIABLES
#######################################################################
# A configuration file can include variable definitions of the form
# var=value
#
# Any subsequent reference to %{var} will have value substituted in.
# 
# These definitions can be permanently overriden on the command line
# with the option -var VAR=value.  Any attempts to set the variable
# in the script will be ignored (with an warning message).  
#

#######################################################################
# Special variables
#######################################################################
# 
# TIME=, $host, %{time}, %{DIR} are reserved and defined as specified 
# in appropriate sections below.
#
# HOMEDIR is also required to be defined.  It will be used as the
# location to put the scripts and their output files in on the remote
# machines.

#######################################################################
# COMMANDS:
#######################################################################
# commands take the form of:
# hostname_list: command
#
# the specified command is put into the scripts for each of the
# machines specified in hostname_list.  
#
# hostname_list is a list of machine names separated by commas like:
# howard138,lovey138,speedy138,petunia138: %{tmodel} %{tmodel_config} >& %{tmodel.out}
#
# tmodel, tmodel_config, and tmodel.out would be set to the full path
# names of those programs and files.

#######################################################################
# TIME=
#######################################################################
# the TIME= command sets the time at which subsequent commands
# execute.  TIME can be set to an absolute time or an expression,
# usually relative to the previous time (%{time}, see below). 
# This time is used to determine how long the script should sleep between 
#
# The commands must be ordered so that time is advancing through the
# file.  Going backward in time results in an error.
#
# NOTE: Most commands should include the '&' background character and
# be terminated with a 'kill program' (e.g. end) at the appointed
# time.  The time command is a bit misleading in that it simply
# determines the amount of time to sleep between subsequent commands
# on the assumption that all commands take approximately zero time to
# run (thus the need to use background jobs) and subsequent kills.
#
# %{time} is a special variable that matches the last value which
# TIME was set to in the script.  It can be used to make the next time
# an offset of the previous time (or to checkpoint specific times)
#
# $host is a special word in the file which is replaced with the 
# machine name when it is put into a script.

#######################################################################
# DONE
#######################################################################
#
# the DONE keyword specifies that all that follows should be inserted 
# directly (with variable substitutions) into the runrunrun script.
# This is usually commands to postprocess data on the local host.  
#

#######################################################################
# @include filename
#######################################################################
# the filename is included in the script as if those lines of text
# were in the file.  Good use of variables (and overriding them)
# should combine with this feature to make building different
# configurations fairly easy, requiring little duplication of effort


#######################################################################
# OPTIONS
#######################################################################
#
# -h, -f filename, -d debug_level, -dir exp_dir, -var var=value, -decode
#
# -h - command line help
#
# -d - set the debugging level to get various extraneous output
#
# -f filename - the configuration file
# 
# -dir - directory to place scripts in, value of %{DIR} (usually used
#        as location to collect data into after the run. 
#
# -var var=value - permanent binding of %{var} to "value".  Any 
#        assignments of var in the input file are overridden by the 
#        command line option.
#
# -decode - DON'T generate an experiment.script file.

#######################################################################
# GENERATED SCRIPTS
#######################################################################
#
# runrunrun - 
#
# The main script generated by this tool is the runrunrun script.
# This script is run on the "master" machine and it copies scripts to
# each of the machines involved in the experiment and launches them,
# waits for them to finish, and collects their output and copies it
# back to the selected directory.  Finally it runs the postprocessing
# specified in the "DONE" section of the configuration file.
#
# run_exp.hostname - 
#
# These are the scripts for each individual machines.  The consist of
# the list of commands to execute separated by appropriate sleep
# commands to delay execution according to the original script.
#
# experiment.script - 
#
# This is the "translated" script with absolute times and complete
# resolved variables.  Useful for debugging or reviewing an
# experimental setup.  Only generated with the -decode option on.


Other DiRT documents
Author: Michele Clark