The model which PEST or PEST++ must run many times is usually much more than just a simulator.

Generally a simulator must be preceded by at least one program that converts parameter values that are calculated by PEST into arrays of numbers that make sense to the simulator. For example, a program such as PLPROC must read a PEST-written file that contains parameter values at the locations of pilot points. It must then undertake spatial interpolation from these points to the cells of a model grid. These interpolated values must then be written to a file that the model reads.

After the simulator has run, programs such as OLPROC, MOD2OBS or BUD2SMP may read simulator binary output files. They may undertake spatial and temporal interpolation of simulator-calculated quantities to the times and locations at which field measurements were made. Of the millions of numbers that a model records, these are the only ones that PEST cares about.

Perhaps the model that PEST or PEST++ must run is even more complicated than this. Perhaps a recharge simulator such as LUMPREM precedes a groundwater simulator. Perhaps a simulator postprocessor calculates electrical conductivity values from simulator-calculated solute concentrations, and then another program calculates items of geophysical interest from these - for example voltages or electromagnetic fields that would be induced by electric currents that are injected into the ground or flow in loops. This may allow the model to be simultaneously history-matched against measurements of groundwater head, salinity and geophysical response. The possibilities are endless. PEST must be able to accommodate them all.

All executable programs which comprise a model must be cited in a batch or script file. When PEST runs the model, it sends a command to the operating system to run this file.

A model that is run by PEST as a batch or script file.

Every time that PEST runs a model, it must provide it with the values of parameters that the model must use on that particular run. It does this by writing these values to pertinent model input files. In doing this, PEST must respect the model's expectations of the content and protocol of these files.

After the model has run, PEST must acquire the values that the model has calculated for quantities that correspond to field measurements of system state and flux. These are recorded on model output files. PEST must know where to find these numbers.

We will address both of these issues shortly. In the meantime we point out that this mode of PEST-to-model communication requires little of the model. A model that works with PEST should not therefore require re-compilation. It must simply be itself. In other words, the PEST-model interface is non-intrusive.

This has some repercussions. Some of these are good and some are bad.

A good repercussion is that a modeller can easily alter the model, for example by adding a new preprocessor or postprocessor here or there. Flexibility in definition of model parameters and model outputs can assist the history-matching process a great deal.

A restriction on this mode of PEST-to-model communication, however, is that model input/output files must be text files, and not binary files. Actually, some of its input and output files can be binary, but not the ones that PEST writes/reads before/after the model runs.

Another restriction is that model input and output protocols should, ideally, be such that parameters and model outputs are recorded with many significant figures. At the same time, the passing of numbers between preprocessors and simulators, and between simulators and postprocessors should preserve all of the precision of these numbers. This ensures integrity of finite difference derivatives calculation.

A model, together with the input and output files that PEST uses to communicate with it.

The protocol for a model input file is set by the program that reads it. This file may contain the values of model parameters. It may also contain other numbers and information that is required by the model. All aspects of this file must be correct when it is read by the model. However the parameters that are recorded in this file must be those that PEST wants the model to use on a particular model run.

The first part of a simple, example model input file is shown below. Parameters are shown in blue. Other text that the model reads from this file is shown in black.

A model input file.

It is easy to make a PEST-useable template of this file. See the following picture. The places that were occupied by parameter values are now occupied by so-called "parameter spaces". Within each such space, a parameter is identified by its name. PEST allows parameter names to be up to 12 characters in length, while PEST++ allows parameter names of up to 200 characters in length.

When PEST writes a model input file, it replaces all parameter spaces on a template file with the values of parameters that they cite. In each case, when recording the parameter's current value, it fills up the space with as many significant figures as the space allows. Meanwhile, all other aspects of the model input file are kept the same. So, when making a template file from a model input file, make each parameter space as wide as you can. (Fourteen to sixteen characters is good.) But make sure that the model is happy with this. In the end, you must respect the model's requirements for the files that it reads. See the PEST manual for more details.

A template file of the above model input file.

Template files can be easily prepared using a text editor, or programs such as Microsoft EXCEL. Start with a model input file and make the necessary alterations.

Once you have prepared a template file, you can use the TEMPCHEK utility (supplied with PEST) to check for any errors that you may have made in its preparation. TEMPCHEK works at two levels. At the lower level it simply checks the syntax of a template file, and lists the names of the parameters that it finds. At a higher level, it writes a model input file using the template file. If your model can read this input file, then your template file is probably fine.

The first part of a simple, example model output file is shown below. Let's assume that PEST must read the numbers that are shown in green, but is not interested in the other numbers. Notice also that the model has recorded these numbers with a lot of significant figures. This is important for finite difference derivatives calculation. Where numbers are very large or very small, scientific notation should be used to record them so that leading zeroes do not occupy spaces that significant figures could occupy.

A model output file.

The template concept does not apply when reading model output files because the contents of a model output file may change from model run to model run. So a different strategy must be employed to identify numbers that must be read from these files. These files are read using instructions; an instruction file is required for each model output file. The set of instructions that are contained in an instruction file present a kind of roadmap that guides PEST to numbers of interest. Here is part of the instruction file that reads the above model output file.

An instruction file.

Like parameters, each number that is read from a model output file is given a name. These numbers are called "observations" in PEST jargon. For PEST, observation names are restricted to 20 characters in length. For programs of the PEST++ suite, observation names can be 200 characters in length.

Instructions files can be easily prepared using a text editor. However where observations are many, this can get tedious. Programs such as OLPROC and members of the PEST groundwater and surface water utility suites can automate this process. Similar functionality is available in PyEMU.

The INSCHEK utility that is supplied with PEST can be used to check the integrity of an instruction file. Like TEMPCHEK, INSCHEK works at two levels. At the lower level it checks that an instruction file is syntactically correct. When operated at the higher level, INSCHEK reads a model output file using an instruction file, and lists the numbers that it reads from that file. This allows you to ensure that the instruction file reads the model output file correctly.

The non-intrusive interface between PEST/PEST++ and a model is summarised in the following figure.

PEST's non-intrusive interface with a model.

So whenever PEST must run a model, it does the following.

•First it writes all model input files on which parameters reside using templates of these files to guide it. These input files then contain the parameter values that PEST wants the model to use on that run.

•Once this has been done, PEST tells the computer to run the model.

•When the model has run to completion, PEST reads observations from model output files using instruction files.

This cycle may be repeated hundreds, or even thousands, of times.

When assembling the model sandwich, a few basic steps can ensure that all goes according to plan.

On every occasion that PEST runs a model, before it actually issues the system command that initiates the model run, PEST deletes all model output files that it knows about. These are the files for which there are complementary instruction files. If the model fails to run, this measure prevents PEST from reading an old model output file, thinking that it is reading a new one. This would lead to disastrous results, including the false idea that at least some model outputs are insensitive to all parameters.

If the model sandwich is complicated, then there are some other quality assurance measures that you can take.

Suppose that one of the executable programs that comprise the model fails to run. Or suppose that this program crashes when it receives a set of inputs that it does not like. Its output file will not be written under these circumstances. Suppose that PEST does not read this output file, but that another model component in a chain of executable programs reads this file. This program must be prevented from reading the old model output file, thinking that it is a new one.

This state of affairs can be prevented if the batch or script file that comprises the model commences with commands that delete all files that are written by one program and read by another. See the example below. Any program that attempts to read a non-existent file therefore crashes. At the end of the model run, there will then be no files for PEST to read. It will know that the model run has failed. (Note that there is no need to delete the final model output files that PEST reads. As stated above, PEST deletes these files itself.)

A model batch file in which the first few commands delete files that are written by one program and read by another.

The above example demonstrates some other useful tricks.

If a program asks a user for information, and then reads its input from the keyboard, keyboard responses can be recorded in a file. When the program is run through a batch file, it can be told to read this file instead of looking to the keyboard. This is done using the "<" symbol.

If a program records information on the screen as it runs, this information can be redirected to a file using the ">" character. It can be directed to nowhere using "> nul" on Windows and "./dev/null" on Linux. This keeps the screen uncluttered as the model runs.