This seems like a good place to draw your attention to some of the things that PEST and PEST_HP can do that you may not have been aware of. Variables which govern various aspects of its operation occupy different sections of the PEST control file. Some aspects of this functionality are also discussed in other of these help pages. See the manuals of PEST and PEST_HP for complete descriptions.

The simplest way to use PEST is to solve a well-posed inverse problem (perhaps rendered well-posed through manual regularisation). Run PEST in "estimation" mode by setting the PESTMODE variable in the "control data" section of the PEST control file to "estimation".

This was the original mode of PEST operation. However, it is a little delicate. If the inverse problem is ill-posed, then PEST will be asked to invert a non-invertible matrix. At worst, this will not work. At best, it will work, but there is no guarantee that the solution to the inverse problem will yield predictions of minimum error variance.

By including a "singular value decomposition" section in the PEST control file you can guarantee that PEST will work even if the inverse problem is ill-posed. It is a good idea to do this. However, because use of singular value decomposition (i.e. SVD) introduces regularisation to the inverse problem, PEST does not calculate a parameter covariance and related matrices when the inversion process is complete. PEST's calculation of these matrices is based on the premise that the inverse problem is well-posed. If it is ill-posed (or runs the risk of being ill-posed), there are better ways to calculate these matrices.

In the old days, users were encouraged to employ PEST's "automatic user intervention" functionality to forestall numerical problems that arise from noninvertibility of critical matrices. However while this functionality is still available, there is little need to use it now, as singular value decomposition does a better job of maintaining numerical stability.

If you suspect that an inverse problem may be ill-posed, it is also a good idea to employ Tikhonov regularisation. This is easily done using the ADDREG1 utility.

LSQR is an iterative solver for large matrix equations. It mimics the action of singular value decomposition (SVD), but is not as exact in partitioning of parameter space into two orthogonal subspaces. However it can be substituted for SVD when parameter numbers are large because of its faster solution speed.

Another means of speeding up solution of the matrix equations that attend highly-parameterised inversion is to use the MKL versions of some of the programs and utilities that are supplied with the PEST suite. These have the same name as the regular versions of these programs, but have an MKL suffix.

An iteration of the inversion process is divided into two parts. First a Jacobian matrix is calculated. Then parameter upgrades are calculated from this Jacobian matrix using different values of the Marquardt lambda. These upgrades are then tested.

For PEST the Marquardt lambda selection and testing process is sequential. For BEOPEST it is partly parallelised. For PEST_HP it is fully parallelised.

If Broyden Jacobian updating is not implemented, PEST advances to the next iteration once it has done one round of parameter upgrade testing. The starting parameters for this new iteration are the best parameters that it achieved during the previous iteration. However if Broyden Jacobian updating is invoked, PEST improves the Jacobian matrix using what it learned from model runs based on different parameter upgrades. PEST learns as it goes. PEST_HP, however, undertakes a second round of parameter upgrade testing based on another set of Marquardt lambdas.

Where an inverse problem is nonlinear, and/or where model numerical problems may degrade continuity of model outputs with respect to parameters, Broyden Jacobian updating may make the difference between good PEST performance and bad PEST performance. This is especially the case when using PEST_HP.

To switch on Broyden Jacobian updating, set the JACUPDATE variable in the "control data" section of the PEST control file to 999. If using PEST_HP, set the UPTESTMIN control variable to at least 25. (This variable is also located in the "control data" section of the PEST control file.) This ensures that PEST_HP calculates a sufficient number of Marquardt-lambda-based parameter upgrades to make updating of the Jacobian matrix worthwhile.

By far the largest numerical cost incurred in solution of an inverse problem is that required for filling of a Jacobian (i.e. sensitivity) matrix. This is normally accomplished using finite-difference derivatives. Each parameter is varied in turn, and then the model is run. Model output differences are divided by parameter differences to obtain approximate derivatives. Obviously, the larger the number of estimable parameters, the larger is the numerical cost of filling a Jacobian matrix. If model run times are large, this cost can become overwhelming.

This cost can be reduced by re-calculating sensitivities of only the parameters that "matter" on all but some iterations of the inversion process. Invocation of sensitivity reuse functionality instructs PEST to re-calculate sensitivities of those parameters whose composite sensitivities are above a certain threshold, except during certain iterations of the inversion process when sensitivities are re-calculated for all parameters.

In certain calibration circumstances, the structure of the Jacobian matrix is such that if two parameters are simultaneously incremented the effects of each of them on model outputs can be distinguished from each other (or almost distinguished from each other). They can therefore be varied at the same time when calculating finite difference derivatives. The number of model runs that is required to fill the Jacobian matrix is thereby reduced.

Simultaneous increment functionality is available only through PEST_HP.

A Jacobian matrix whose structure permits simultaneous increments.

PEST_HP can save model runs by calculating a rank-deficient approximation to a Jacobian matrix; this is similar to what ensemble methods do. Sensitivities of model outputs with respect to individual parameters are back-calculated from empirically-determined covariances. These can be calculated using only a few hundred model runs per iteration, regardless of the number of parameters that feature in the inverse problem.

Optionally, localisation can be used to increase the rank of the Jacobian matrix. As the inversion process progresses, the rank-deficiency of the Jacobian matrix is further reduced. This is because different random parameter increments are employed during each iteration of the inversion process. Meanwhile, part of the Jacobian matrix from previous iterations can be retained in the Jacobian matrix which is employed to calculate parameter upgrades during the new iteration.

The model run efficiency of this process can be very high. Randomised Jacobian calculation can be used in conjunction with Tikhonov regularisation. Experience suggests that it can work well in detecting narrow zones of connected permeability. This is demonstrated in a GMDSI worked example.

SVD-assist is supported by both PEST and PEST_HP. It works like this.

A modeller prepares for highly-parameterised inversion in the usual way. This requires development of the model sandwich, the writing of appropriate template and instruction files, and the construction of a PEST control file. Normally Tikhonov regularisation is featured in the PEST control file.

The model is then run once for each parameter, and a Jacobian matrix built. Using singular value decomposition, so-called "super parameters" are then defined. These span the solution space of the inverse problem. Next, a new PEST input dataset that is based on these super parameters is built by running the SVDAPREP utility. The inverse problem is then solved using super parameters. As the number of super parameters is normally far smaller than the number of actual model parameters, the model run burden of highly-parameterised inversion is reduced considerably.

SVD-assist works well in many circumstances. However its performance can be compromised by model nonlinearity. Formulation of super parameters may then need to change as parameters are updated.

SVD-assisted inversion.

We'll have a little more to say about SVD-assist in a later section.

The model that is run for calculation of finite-difference derivatives with respect to some or all parameters does not need to be the same model that is used for testing of parameter upgrades. The former may be a simple, fast-running version of the latter that has sufficient complexity for integrity of calculation of at least some derivatives. Meanwhile, the integrity of parameter upgrade testing is safeguarded through use of a more complex model - the same model that is used to calculate objective functions, and the same model that will ultimately be used to make decision-pertinent predictions.

While the ability to use multiple models in this fashion seems simple, in fact it requires a considerable amount of behind-the-scenes book-keeping. It may also require some strategic transfer of files between model run agents and the run manager (if model runs are parallelised).  

To find out more about these options, look for the following topics in the PEST and PEST_HP manuals:

•multiple command lines;

•user-prescribed zero sensitivity;

•observation re-referencing;

•model file distribution.