On Feb 10, 7:47*am, "Andy M." wrote:
On Feb 9, 11:34*pm, Liam Steele wrote:



On 09/02/12 21:42, Andy M. wrote:

On Feb 9, 9:28 pm, Liam *wrote:
I know this forum is populated by various people with meteorology
backgrounds, so thought I'd ask here first...

For my PhD I am assimilating data into a general circulation model. Now
what I want to do is compare the assimilation result (a simple lat-lon
plot of whatever) with a lat-lon plot of the observations.

I can simply subtract one from the other to get a difference plot, but I
was wondering if there was a statistical way to compare the 'fit'
between two 2D plots? The model calculates the average rms error over
all lat-lon points, but this isn't very helpful, as one particularly bad
gridpoint can skew the whole result.

If you can give any helpful advice I shall endeavor to mention you in
the acknowledgements of my thesis. )

Many thanks,

--
Liam (Milton Keynes)http://physics.open.ac.uk/~lsteele/

Why not treat the assimilated version as a forecast? *You can
construct various forecast scores averaged over spatial zones. *We
often use ROCs but it depends what you want to test. *Anything you
might do with a time series i.e. *sequence of forecasts you can use
for a single event over space. You could use this to test the
improvement between different assimilation methods and different
assimilation starting points. *I have no direct experience of working
in assimilation schemes but we look for skill in much that we do.

There is a good page on verification here.

http://www.cawcr.gov.au/projects/verification/

Andy

Further reading of the web page has brought up the CRA verification
method, which compares spatial details as well as magnitudes. And, it
even has some IDL code supplied! Looking good...

http://www.cawcr.gov.au/projects/ver...ification.html

--
Liam (Milton Keynes)http://physics.open.ac.uk/~lsteele/

Yes it is a good site .., I have not used those scores but having
some *idea of the spatial coherence and the departure from the target
is good ... might look myself.

You should also think about getting hold of a copy of Jolliffe and
Stephensonhttp://books.google.co.uk/books/about/Forecast_verification.html?id=Q...
I am told by Dave that a new edition is on its way.

What model are you building the assimilation system for?

Good Luck,

Andy

I am working on a model of the martian atmosphere, and we are
currently using the 'Analysis Correction' assimilation scheme that
they used at the Met Office in the early 90s. There are more modern
assimilation schemes, but apparently they have difficulty when only
assimilating a few observations (and for Mars we only have one sounder
that gives us observations). I'm looking in particular at water vapour
and ice data, and trying to get the model's cloud prediction as
accurate as possible so we can work out what role they play in Mars'
climate.

As an example, I've put three plots up on my website:
http://www.physics.open.ac.uk/~lsteele/assimilation.png. The top panel
is observation data, the middle panel is an assimilation and the
bottom panel is a standard model run. To my eye the assimilation
resembles the observations more than the standard model, but it
actually has a larger rms error. Hence I was looking for something
that weights things spatially as well as simply by magnitude.

Liam