• 7-18-06: Converted env_readme.txt into this htm file.

• Introduction
• Relevant files
• Required apps
• How environmental plot generation works
• How archiving works
• How the scripts are run automatically by cron
• Detailed description of the innards of each script
• How to generate environmental plots manually
• How to update RTD/POWER mappings for plots

We have written a series of scripts and programs to get the houston control environment log files (housctl*.log) and create plots of the various parameters for viewing on the web.

Currently, these scripts reside and function on Cocoa in /home/environment. An identical ccopy should also be kept on Grlab in /home/apollo/src/environment. The scrips are run by a cron job. They get the logfiles from Houston, which is mounted on Cocoa, generate plots (in the form of png image files) and web pages, which are viewable from /home/www/html/environment.

We also have scripts that archive the plot images. They copy the plot images and put them in /home/www/html/environment/archive. These scripts are also run by a cron job.

• Run update.script on Cocoa
  This shell script automatically launches the other scripts and updates the environment plot webpage. The following is pretty much a line by line explanation of what's going on.
• cd /home/apollo/environment
  This makes all the scripts run in the '/home/apollo/environment' folder. This is what I refer to in this help file as the 'local folder'.
• getlog.py
  This looks at houston, figures out the name of the newest housctl*log file, then uses 'scp' to copy the file to the local folder on the Cocoa. It used to (but no longer does) take the 2 most recent logfiles and stick them together with the 'cat' command in a file called 'current_log.log'.
• ./logclip.py 1.5 current_log.log
  This creates a file 'current_log.log' that contains the most recent 1.5 days worth of housctl*log records. It currently assumes the logfiles are contained in the '/home/apollo/environment/' folder.
• ./env_parse.py current_log.log
  This will parse the file 'current_log.log' (in the local folder) for parameters like the gps dac, RTD temperature readouts, flow rates, etc. It outputs the several whitespace delimited data files (into the local folder) : 'env_control.dat' 'env_dac.dat' 'env_flow.dat' 'env_temps.dat'.
• ./env_plot.py -dAgg
  This uses dataparse.py to read-in the 'env*dat' files created above. It then uses matplotlib to generate plots 'env*.png' that are png image files. The '-dAgg' option makes matplotlib use the 'antigrain geometry backend', which doesn't require an X-Display, and outputs pngs.
• ./env_webpage.py
  This simply generates the htm webpages that you can view the plots in: plots-overall.htm, plots-temps.htm, and plots-flow.htm.
• And the final commands in update.script just move the pngs and the htm files to the appropriate location for viewing on the WWW.

This is a Python script that copies the image plot files to the archive location, and generates the html files for the archive pages.

At the top pf the script, there are two strings defined. One is 'archiveloc' -- this is the folder you want the archive images and associated htm files to be put. The other is 'imagesloc' -- this is the folder that the plot images that you want to archive are located. I wrote the script to use absolute paths, so you should probably use absolute paths ('/home/environment' okay, '../environment' not okay). The script should run properly regardless of what folder it is run in. I think.

Here is how the archives are organized. Within the 'archiveloc' folder, there will be a folder for each month, like "0601/" for Jan 2006. And inside each folder are plot images for each day, like "060120env_temps.png" for a plot copied on Jan 20, 2006. In the 'archiveloc' folder, an 'index.html' file is created that has links to all the htm files for each month. And in each month folder there is an htm file, like "0601/0601.htm" that has links to the plot images in that folder.

To get lists of folders, and copy files around, we use the os.popen() function in the os module.

Here is a step by step rundown of the algorithm:

• Get a list of all the folders in 'archiveloc'. And if this month doesn't have a folder, create one. So the first time the script is run in July 2010, the script will create a folder called "1007/" in 'archiveloc'.
• Get a list of today's plot images located in 'imagesloc'. The script assumes that the plot images are png files, and end in the 'png' extension.
• Copy today's plot images to the appropriate folder, and add the date to their filename. So 'imagesloc/tst.png' would be copied to archiveloc/0607/060710tst.png' if the time the script ran was July 10 2006.
• Generate the 'archiveloc/index.html' page. This is done by the 'makeindex' function. It is passed a list of all the folders in 'archiveloc', like ["0604/","0605/",...]. It generates a table with link to the htm files inside each folder.
• Generate the htm files for each folder. In other words, generate 'archiveloc/0607/0607.htm'. This is done by the 'makemonthpage' function. You pass it a string of the folder you want to generate the htm file for. The script only calls the 'makemonthpage' function for the 2 most recent months, since there is no point in regenerating htm files for older months. This function uses the calendar module to know what day of the week a date like July 12, 2006 is.

Here is what my crontab looks like. The plots are generated once every 5 minutes, and they are archived once a day.

SHELL=/bin/sh
HOME=/home/aorin
PATH=/usr/local/bin:/usr/bin:/usr/sbin:/bin:/sbin:/usr/X11R6/bin:.
LOGNAME=aorin


*/5 * * * * /home/environment/update.script
0 0 * * * /home/environment/env_archive.py