Skip to main content


Showing posts from October, 2015

High performance computing: how to run IDL program on a cluster

One of the advantages of IDL is that it could be deployed on Linux system, to say a HPC (High Performance Computer). Fortunately my research group have this computational resource to advance my research.

So the question is how to run IDL programs on HPC?
Basically of course you can use X-Windows and IDLDE like what you did in Windows system.
But that is NOT what I am about to cover in this discussion.

Before I start, you properly could refer to this page first.
Some core idea is as follow:

: The above line tells Linux to use the shell /bin/bash to execute
: this script. That must be the first line in the script.
: You must have no lines beginning with # before these
: PBS lines other than the /bin/bash line:
#PBS -N 'hello_parallel'
#PBS -o 'qsub.out'
#PBS -e 'qsub.err'
#PBS -W umask=007
#PBS -q low_priority
#PBS -l nodes=1:ppn=4
#PBS -m bea

: Change the current working directory to the directory from which you ran qsub:
: Run IDL …

Parameter estimation: data format in model I/O

Parameter estimation, also referred as model calibration, usually involves with model simulations with different sets of data and parameters.

A common practice is to replace the parameter within model inputs with different values and calculate the minimum objective function, which is used in PEST.

Most of the time, parameter estimation can be very computational. Therefore, a better data format can reduce I/O burden and improve the efficiency.

A typical example is as follows:

To calibrate the groundwater modeling such as MODFLOW, a few template files much be prepared for PEST package. Since PEST requires that all ASCII file must NOT exceed 2000 in width. This means that all the template files must be written with care.
For simulations with large spatial domains, the input files are usually large as well. And it is very important to keep the data format well designed that the accuracy and efficiency could be compromised.

On the other hand, PEST also reads the output from model simulatio…

Integrated groundwater and surface water modeling: topography effects

This is a talk highly related to my last post on spatial discretization of numerical simulation.

In groundwater modeling such as MODFLOW, topography effects are very common and yet less discussed.

There are a few distinguishable differences between low land groundwater and high land groundwater system. First, heads at high elevation are usually higher, this is observed since water table usually follow topography even not strictly. Second, variances in heads in high elevation may be insensitive to changes in forcing, For example, the low land always receive the most water (even flood) and droughts. Third, generally, water movements in high elevation are much slower due to low K values.

With consideration of these differences, it is easier to understand a few critical phenomena in groundwater modeling.

First, groundwater system is seldom in steady state. The best estimation, however, could be within winter time when stream base flow is minimal. At this stage, only limited groundwater fl…

Integrated groundwater and surface water modeling: spatial discretization

Numerical simulations have to break time and space into increments. For example, weather predictions are usually modeled in hours or days and maps are produced with resolution.

In contrast, our real world is always behaving continuously. For example, temperature always evolves with time continuously.

We are also interested in a three dimensional simulation instead of 1D or 2D. The real world itself is 3D though we constantly forget that. It is just doesn't benefit much if we had a 3D temperature map.

For other types of simulation, however, 3D could provides much better solutions.
Groundwater flow, as an example, is best simulated with 3D numerical simulation.

The spatial discretization is always a challenge for groundwater modeling. Horizontal and vertical discretization are always coupled together. For example, in a flat grassland with homogeneous hydrologic proprieties, the horizontal discretization becomes less important. However, if the surface elevation changes drastically, the h…

Integrated groundwater and surface water modeling: restore the stream index relationship

Numerical hydrology models usually require the stream index to be built prior to the actual simulation.
It is also good practice to keep them in order so it is much easier to analyse the results.

Here I have provided an examples how it could be done using some scripts and ArcMap.
After the drainage line is defined from watershed delineation process such as using Arc Hydro tool.
The resulting attribute table will be like this:

First, it is good practice to make a copy of the file. Then we need to "Delete" unwanted fields and only keep the HydroID, GridID and NextdownID fields. Then export the attribute table into a text file and run the IDL routine "restore_drainageline_spatial_relationship", which will produce a new index for each stream segment. You can copy the created text file content into the Microsoft Excel for next step. Then edit the drainage shapefile feature and replace the HydroID and NextdownID fields using the just copied indices in the Excel. After th…