SUBJECTIVE
LIST OF MODEL PERFORMANCE CHARACTERISTICS
1.
Introduction The NCEP model suite is upgraded numerous
times annually. Since each "model" is actually a system of integrated
components, even slight �tweaks� to any of the components can drastically
effect the model's performance characteristics. For example, the
Global Forecast System (GFS) consists of the initialization scheme (Global
Data Assimilation Scheme - GDAS), the Global Model itself, and the post
processed grids that are made available for use in grid and grib format.
Even a slight modification to any one of these components can drastically
effect the perceived performance of the model. As a result it
is not only difficult to isolate consistent model performance characteristics
(loosely referred to as �bias�) across the model upgrades, but also the
source of the bias.
Examples of modifications to a model
system that can effect model performance characteristics include:
Modification of initial data ingested by model
(volume, type, density, accuracy)
Modification of model structure (horizontal/vertical
resolution, time step increments, domain, grid point vs spectral wave model,
vertical coordinate, hydrostatic vs non hydrostatic models)
Physics packages used by the model (radiation
schemes, diffusion, land/surface representations)
Parameterization schemes used by the model
(convective parameterization, frictional components)
Post Processing of model data (precip type
algorithms, resolution of the grid the model data is displayed upon)
2.
Model Performance Characteristics Model performance is a function of model
error - which can be split into two components (systematic error
and random error).
The random component of error is that
which we are unable to easily attribute a cause (and therefore not
easily correct).
Systematic error can be automatically
removed from model output to correct or minimize the amount of error in
the model solution.
A subjective list of model performance
characteristics (biases) have been obtained via forecasters and is available
in the next section. This information is useful for both forecasters
and by the modelers.
The majority of the biases listed below
are from WPC. However, in order for EMC to gain a more complete picture
of model performance, all users of NCEP model output should provide their
subjective observations of model performance by submitting
their subjective observations of bias.
This information will be conveyed to WPC
and EMC and potentially added to this web page.
5.
Other model information Want to know the latest upcoming plans
for model development and system resources as it relates to modeling at
NCEP ???
Click this link /html/model2.shtml#synergy
Over predicts QPF at times in regions
of moderate synoptic scale forcing
Southeast and east coast
Anytime
NCEP WPC
2000
.
Initial Analysis Scheme ?
CMC
GEM
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.
CMC
Global
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.
ECMWF
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.
NAM
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
GFS
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
GFS too ambitious with strength and speed of systems crossing Sierras after fhr 36
SW US
Cool season
USGS
Dec 2005
Too progressive and strong with systems crossing Sierras
Model resolution of topography
Convective Feedback
Primarily east of front range and
west of Appalachians
Warm season, any time of model day
NCEP WPC
Spring of 1998
When specific thresholds in the mass
fields are met, convective scheme is triggered and then dumps a large amount
of QPF over a grid point - releasing so much latent heat over the grid
point that the model is forced to adjust the mass fields by producing a
local vertical motion max in the mid troposphere (~ 500mb), a corresponding
upper level jet max over the vertical motion max - an intense and small
scale vort max in the mid levels (MCV).
The model scales up the mesoscale circulation
at mid levels and holds onto it as a real feature for as long as 3 days.
The model can produce precipitation in
association with the feature as it tracks along in the flow.
GFS Convective Parameterization Scheme
Dry bias north of areas where over
2" of QPF has been produced in a 6hr period
Primarily east of front range and
west of Appalachians
Warm season, any time of model day
NCEP WPC
Spring of 1998
QPF produced from convective feedback
blocks northward advection of moisture
Result of GFS Convective Parameterization
Scheme
QPF verification historically better
than Eta
CONUS
Cool Season only
NCEP WPC
1999
Rely more heavily on QPF from GFS
- especially beyond 36 hours
GDAS better than EDAS ?
Aerial coverage of QPF and mass fields
over done (QPF at low thresholds .01" and .10")
CONUS
Anytime
NCEP WPC
Since mid 1990's
Over forecast of aerial coverage of
precip can lead to high bias in PoPs
Model resolution (the lower the resolution
the more geography a QPF pattern can get spread over)
Slightly ambitious with magnitude
of high amplitude patterns
North America
Cool season so far
NCEP WPC
Since fall 2002
Prediction of southward progression
of cold air over done
Model a bit too extreme in temp patterns
beyond 84 hours
Precip Type Algorithm off of GFS
too eager to depict snow
?
Ambitious to phase northern and southern
stream systems in fast and spit flow patterns beyond fhr 84
North America
Cool season
NCEP WPC
Cool season 2001 (not noticed yet
in 2002)
Over forecast of cyclogenesis east
of Rockies
Suspect related to model resolution
and lack of dense obs data where associated systems originate in forecast
cycle
Major difference in QPF forecast than
ETA
CONUS
Warm season
NCEP WPC
Since mid 1990's
Lack of run to run continuity in QPF
Different convective parameterizations
between models result in different QPF forecasts (primarily in areas where
synoptic scale forcing is weak)
NOGAPS
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
Prior to recent upgrade was typically the
model that exhibited lowest AC scores.
Seemed very angular in its depiction of
500mb heights (extreme gradients beyond 48 hours).
.
.
.
.
.
.
RUC
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.
UKMET
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Seems better than GFS with forecast
of phasing of systems in northern and southern branch of jet
North American middle latitudes
Anytime
NCEP WPC
Since fall of 2001
When GFS is showing phasing of systems
beyond 84 hours, check UKMET to see if solution is consistent
GDAS ?
ENSEMBLE
PREDICTION SYSTEMS
CMC
EPS
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.
ECMWF
EPS
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.
NCEP
GFS EPS
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Ensemble mean maxima/minima much more
subdued compared to actual verification
Anywhere
Anytime
NCEP WPC
Fall 2000
Position forecast decent, but actual
strength of a system watered down
This is normal for ensemble output
with significant spread as it takes the average of numerous solutions thereby
flattening the amplitude of "ridges or valleys in mass fields
The ensemble mean does well, in situations
where operational GFS is a bit too aggressive, the GFS ENS MEAN seems to
verify better at 500mb beyond 96 hours.
North America
Anytime
NCEP WPC
Fall 2001
More run to run continuity in EPS output, than operational runs.
Because EPS output is average of numerous solutions, it is less
sensitive to extreme solutions.
NCEP
SREF EPS
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Ensemble mean maxima/minima much more
subdued compared to actual verification
Anywhere
Anytime
NCEP WPC
Fall 2000
Position forecast decent, but actual
strength of a system watered down
This is typical for ensemble output
with significant spread as it takes the average of numerous solutions thereby
flattening the amplitude of "ridges or valleys in mass fields
NOGAPS
EPS
Subjectively Observed
Bias
Geographical location
of bias
Annual/Diurnal
attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.
Submit
a bias Fill out the form and click SUBMIT and it will be reviewed by WPC personnel.
We will then need to contact you for more specifics, so please leave
an email address.