10.15.2 Output Options: The number of decimal places

Under certain options, the user has the choice of the number of decimals to be used and whether or not the data is output in decimal format or E-formats (e.g., 1.23 as opposed to 0.123E+01).  These choices relate to questions of “precision” and/or “rounding errors”.

Problems of a loss of precision may arise if an insufficient number of decimal places are not used due to rounding errors. For example, a trip matrix cell value 10.12345 would be output as 10.12 if the output format specifies 2 decimal places; in effect 0.00345 trips are lost. If the number of decimals were 5 no trips would be lost.  The losses may become significant in very large matrices where a large number of cells have been “in-filled” with very small values.

Thus extra decimal points may be needed for extra precision, although this may also depend both on the format used and on the “units” of the matrix. For example (assuming “decimalised” outputs) if a matrix contains inter-zonal travel times which are of the order of 30 minutes in units of seconds 2 decimal places may give ample precision (30 minutes would be output as 1800.00 seconds), but if the units were hours then 30 minutes becomes 0.50 on output and any O-D cell less than, say, 0.01 hours (36 seconds) might be rounded down and output as 0.00 under 2 decimals.  Increasing the number of decimals avoids such problems.

On the other hand using a large number of decimal places may create problems in certain circumstances, e.g., CSV outputs (see above), where the length of an individual record may exceed 16384 characters and be truncated. Compromises on the number of decimals may sometimes be necessary.

However we note that trailing zeros (and/or the decimal point) are truncated. Thus 10.10 becomes 10.1 on output as CSV, 10.00 becomes simply 10 etc. etc. Thus increasing the number of decimal points does not necessarily increase the size of the CSV output records proportionately.

Note that the default number of decimal places is now 5 but it may be re-set via the namelist parameter NDPS used in the MX preferences file mx0.dat.