Out of curiosity, is there a "grade" assigned to judge the accuracy of a
storms forecast?

What I'm think, is that we all talk about the weather....hurricanes/tropical
cyclones....but is there an objective measurement of how well the
forecasters did?

Since I'm in the USA, I'm specifically thinking of the US NHCs forecasts. I
know they use many models and then make a judgment as the which they think
is the best. From that they issue their "official" forecast every 6 hours.
But how do they measure their success? Could we grade their forecasts on
Charlie as a C+ and Francis as a B?

I'm working on some ideas and wondering if this has already been done.

Alf
Marietta, GA, USA

http://www.nhc.noaa.gov/pastall.shtml about 1/3 of the way down the
page. They've neglected to post verifications for recent years, or if
they have, they've posted them some place I can't find.

Also: http://www.tropicalupdate.com/2003%2...rification.htm



On Tue, 21 Sep 2004 11:55:59 GMT, "Alf Sauve" wrote:

Out of curiosity, is there a "grade" assigned to judge the accuracy of a
storms forecast?

What I'm think, is that we all talk about the weather....hurricanes/tropical
cyclones....but is there an objective measurement of how well the
forecasters did?

Since I'm in the USA, I'm specifically thinking of the US NHCs forecasts. I
know they use many models and then make a judgment as the which they think
is the best. From that they issue their "official" forecast every 6 hours.
But how do they measure their success? Could we grade their forecasts on
Charlie as a C+ and Francis as a B?

I'm working on some ideas and wondering if this has already been done.

Alf
Marietta, GA, USA

So they've got it covered with lots of analysis. I didn't realize that was
where the NHC did post mortem, because when I clicked on 2004, it said that
a chart would be coming after the season was over.

I guess if I want to verify the numbers I'll have to wait until the end of
the season and compare against NHC analysis. (I'm too lazy to run the
numbers on a previous year's storm.)

Thanks, Alf


"The Artist Formerly Known As Your Highness" BurnTheAstroCharts wrote in
message ...

http://www.nhc.noaa.gov/pastall.shtml about 1/3 of the way down the
page. They've neglected to post verifications for recent years, or if
they have, they've posted them some place I can't find.

Also: http://www.tropicalupdate.com/2003%2...rification.htm



On Tue, 21 Sep 2004 11:55:59 GMT, "Alf Sauve" wrote:

Out of curiosity, is there a "grade" assigned to judge the accuracy of a
storms forecast?

What I'm think, is that we all talk about the
weather....hurricanes/tropical
cyclones....but is there an objective measurement of how well the
forecasters did?

Since I'm in the USA, I'm specifically thinking of the US NHCs forecasts.
I
know they use many models and then make a judgment as the which they think
is the best. From that they issue their "official" forecast every 6
hours.
But how do they measure their success? Could we grade their forecasts on
Charlie as a C+ and Francis as a B?

I'm working on some ideas and wondering if this has already been done.

Alf
Marietta, GA, USA

Alf Sauve wrote:

I
know they use many models and then make a judgment as the which they think
is the best.

I just looked at the "official track" of Jeanne and noticed that it is further
East than any of the models listed at wunderground.com. Four of five show it
hitting Florida while the official track has it hitting NC. I wonder what kind
of judgement they are using to overrule all those models and why that info
can't be incorporated in the models.

Fred

Well if you have numerous models, including I believe the "what happened
with the most similar one like this" model. Someone has to make a judgment
call as to whom to believe. I would believe they keep tracking statistics
on all models and they are aware that some models work better under certain
conditions and others work better other times. So the head forecaster
honcho surely takes all that into account.

But given the "official" forecasts and official track. I'm developing a
"handicap" for each storm. (The lower the number the better.) I rate NHC's
Charlie forecast a 567 - not very dependable-way off course. And Francis a
131 - good-a solid "B" reasonably close not to cause unnecessary panic, but
still not fully predictable.

I want to run the numbers on two more storms, Ivan and Jeanne, to see if my
handicap matches the subjective "feel".


"FHemmer209" wrote in message
...
Alf Sauve wrote:

I
know they use many models and then make a judgment as the which they think
is the best.

I just looked at the "official track" of Jeanne and noticed that it is
further
East than any of the models listed at wunderground.com. Four of five show
it
hitting Florida while the official track has it hitting NC. I wonder what
kind
of judgement they are using to overrule all those models and why that info
can't be incorporated in the models.

Fred

In article .net,
Alf Sauve wrote:
Well if you have numerous models, including I believe the "what happened
with the most similar one like this" model. Someone has to make a judgment
call as to whom to believe. I would believe they keep tracking statistics
on all models and they are aware that some models work better under certain
conditions and others work better other times. So the head forecaster
honcho surely takes all that into account.

And there are also objective techniques such as the superensemble (FSU)
and the NRL systematic approach that do this as well.

But given the "official" forecasts and official track. I'm developing a
"handicap" for each storm. (The lower the number the better.) I rate NHC's
Charlie forecast a 567 - not very dependable-way off course. And Francis a
131 - good-a solid "B" reasonably close not to cause unnecessary panic, but
still not fully predictable.

I want to run the numbers on two more storms, Ivan and Jeanne, to see if my
handicap matches the subjective "feel".

FORECAST ERRORS (KM) FOR CHARLEY
OFCL 67.7 132.6 172.4 190.0 9999.0 307.7 9999.0 746.3 9999.0 1729.9
#CASES 23 21 19 17 0 13 0 9 0 5

FORECAST ERRORS (KM) FOR FRANCES
OFCL 36.4 66.5 94.8 119.3 9999.0 152.9 9999.0 223.9 9999.0 291.8
#CASES 51 51 51 51 0 51 0 50 0 46

FORECAST ERRORS (KM) FOR IVAN
OFCL 43.1 88.5 148.5 214.8 9999.0 341.1 9999.0 473.3 9999.0 578.1
#CASES 58 58 58 58 0 58 0 55 0 52

FORECAST ERRORS (KM) FOR JEANNE
OFCL 49.2 88.3 134.0 181.4 9999.0 297.9 9999.0 473.9 9999.0 690.2
#CASES 35 33 31 29 0 25 0 22 0 18

These errors are every 12 h from 12 h to 120 h.
--
One day, the wind blowed so hard, it blowed a well up out of the ground;
blowed so hard, it blowed a crooked road straight. Another time it
blowed an' blowed, an' scattered the days of the week so bad Sunday
didn't get around 'til late Tuesday mo'nin. - The WPA Guide to Florida

Alf Sauve wrote:


I want to run the numbers on two more storms, Ivan and Jeanne, to see if my
handicap matches the subjective "feel".

Since I live in Florida I hope Jeanne gets a high handicap as a result of her
staying WAY east!

Good luck with your system.

Fred

Super, Sim, thanks.

My data entry and crunching is pretty close to the numbers below, especially
for Charlie. I may have some typos. I'm concentrating on 1, 3 and 5 day.
(Remember the scaling logic used in analog meters?)

Once I have the five major storms done and graphed, I'll post them.

Thanks again, Alf



"Sim Aberson" wrote in message
...
In article .net,
Alf Sauve wrote:
Well if you have numerous models, including I believe the "what happened
with the most similar one like this" model. Someone has to make a
judgment
call as to whom to believe. I would believe they keep tracking
statistics
on all models and they are aware that some models work better under
certain
conditions and others work better other times. So the head forecaster
honcho surely takes all that into account.

And there are also objective techniques such as the superensemble (FSU)
and the NRL systematic approach that do this as well.

But given the "official" forecasts and official track. I'm developing a
"handicap" for each storm. (The lower the number the better.) I rate
NHC's
Charlie forecast a 567 - not very dependable-way off course. And Francis
a
131 - good-a solid "B" reasonably close not to cause unnecessary panic,
but
still not fully predictable.

I want to run the numbers on two more storms, Ivan and Jeanne, to see if
my
handicap matches the subjective "feel".

FORECAST ERRORS (KM) FOR CHARLEY
OFCL 67.7 132.6 172.4 190.0 9999.0 307.7 9999.0 746.3 9999.0
1729.9
#CASES 23 21 19 17 0 13 0 9 0
5

FORECAST ERRORS (KM) FOR FRANCES
OFCL 36.4 66.5 94.8 119.3 9999.0 152.9 9999.0 223.9 9999.0
291.8
#CASES 51 51 51 51 0 51 0 50 0
46

FORECAST ERRORS (KM) FOR IVAN
OFCL 43.1 88.5 148.5 214.8 9999.0 341.1 9999.0 473.3 9999.0
578.1
#CASES 58 58 58 58 0 58 0 55 0
52

FORECAST ERRORS (KM) FOR JEANNE
OFCL 49.2 88.3 134.0 181.4 9999.0 297.9 9999.0 473.9 9999.0
690.2
#CASES 35 33 31 29 0 25 0 22 0
18

These errors are every 12 h from 12 h to 120 h.
--
One day, the wind blowed so hard, it blowed a well up out of the ground;
blowed so hard, it blowed a crooked road straight. Another time it
blowed an' blowed, an' scattered the days of the week so bad Sunday
didn't get around 'til late Tuesday mo'nin. - The WPA Guide to Florida