The Third MOPEX Workshop in Sapporo, Japan, July 7-9, 2003 (HW08 PARAMETER ESTIMATION TECHNIQUES - IUGG 2003)

This was the third Workshop held as part of the World Climate Research Global Energy and Water Cycle Experiment (GEWEX) Model Parameter Estimation Experiment MOPEX) to enable the wider scientific community to address the issue of hydrological model parameter estimation. Over three days 28 presentations were made to this Workshop supported by some six posters. Presentations were under the following sub-themes of (i) Development and application of model calibration techniques, (ii) Review of model parameter estimation experiment workshop results, (iii) Parameter estimation and model diagnostic techniques, (iv) Uncertainties in modelling and information in data and (v) MOPEX's contribution to PUB and MOPEX future plan and direction. In addition to the discussion of each presentation a discussion period was held at the end of each sub-theme to draw out the key issues. The Workshop received strong support with at times over 100 participants to hear the invited speakers and good discussion of the issues. Considerable interest was shown in publishing a special issue of a Journal on Model Parameter Estimation and an accompanying IAHS Red Book.

The final discussion session concerned future activities and support of PUB. It was agreed that a MOPEX-4 Workshop would be held in late 2004, accepting the offer of Cemagref located outside Paris. Data from more basins (100) from different climatic areas would be provided with a subset of these involving soft data for a detailed examination of its use. It was agreed that the Sapporo Workshop did not allow sufficient time to pursue the discussions in depth and that this would be a key feature of this Workshop. To partition the discussion to make it more structured PUB could be considered as the rational for MOPEX. Starting with Information (Soft/Medium/Hard Data) feeding by various methods into the Models (how should we build models, etc?), we need Theory/Empiricism and Philosophy to achieve this and can then consider Applications. From this we have four primary activities. The issues are: (i) How do we bring the information to bear on the models (identification, parameter estimation, data assimilation, evaluation is part of a whole host of issues)?; (ii) What are the right models? (Are they doing the right job for us?); (iii) Generalisation (the application of models to PUB, etc.) and (iv) Assessment of the methods, models and applications. To try and understand the results obtained it is planned to use of Diagnostic Methods akin to the approaches used by the atmospheric and climate modelling community.

The MOPEX-5 Workshop is proposed for the VIIth IAHS Scientific Assembly to be held at Foz de Igassu, Brazil in mid 2005. This Workshop will present the results and findings of MOPEX-4 and will involve new, innovative analyses of the science questions and the application of a priori parameter estimation to a common set of basins over a wide range of different climatic regimes. The Workshop will also include additional contributions on a priori parameter estimation techniques applied to large-scale basin applications and specific regional/country applications where both model developers (modellers) and users (water managers) are involved. Based on these experiences, the discussions will be focused and structured to provide guidance to PUB and MOPEX on a refined set of science issues and provide a way forward to addressing and resolving these.

John Schaake, Yun Duan, Alan Hall

Summary Discussion

Basic Structure of MOPEX

John Schaake

Based on the MOPEX-2 Workshop and the discussion over the last three days the suggested MOPEX science areas should be: (i) Indentification tools, (ii) Models, (iii) Generalisation (of models?), (iv) Evaluation strategy and (v) Diagnostic methods.
(Refer to John's PP presentation if more needed here.)

Hoshin Gupta

It was difficult to pursue the discussions in depth as we pick up on something which connects to something which connects to something else and by the time we are ready to go a little bit deeper it is the end of the discussion period. How can we partition the discussion to make it more structured. We can start with PUB as rational for MOPEX. In the middle we have Models - what are the right kind of models, how should we build models, etc. Below this we have Theory/Empiricism and Philosophy and Applications. Feeding into the Models by various methods is Information (Soft/Medium/Hard Data). From this we have four primary activities. The issues are: (i) How do we bring the information to bear on the models (let's not call it identification, parameter estimation, data assimilation, evaluation is part of a whole host of issues)?; (ii) What are the right models? (Are they doing the right job for us?); (iii) Generalisation (the application of models to PUB, etc.) and (iv) Assessment of the methods, models and application. These four topic areas could form the basis for deeper discussion at future workshops.

John Schaake

I totally agree with this but I have a fifth topic called Diagnostic Methods. If you look at the way modelling is conducted in the atmospheric and climate community they spend a lot of time talking about diagnostics. In hydrology it has not even been our tradition to even think that way. This is an important area and I would like to see we can learn from that and include some other perspectives and concepts that integrate all the way through this.

Siva Sivapalan

The picture put up there of information feeding into models , improving that process, models being generalised, so that they can be used for prediction in ungauged basins. This still relates to Target 1 in PUB - using the current models and seeing how we improve them for prediction in ungauged basins. The whole process of model development is not included in this process as shown. Aside from MOPEX, PUB needs to give emphasis to changing the paradigms and a new paradigm may not work using this framework as presented. A totally different approach may be needed. There are new ways of constructing models, new space-time characterisations of processes.

Hoshin Gupta

Within this framework the methods of assimilating information is not restricted to current methods.. You can come up with new and radically different ways of how you view information and models.

John Schaake

Can we agree on this approach of these four topics with diagnostic methods added?

VJ Gupta.

Picking up on diagnostic methods, this is a fundamental issue in hydrology. Whatever framework you have there should be a framework to falsify your model. This is what diagnostics means. If there is no falsification then we have not learnt anything, then we have predetermined what we have construed and we reproduce that. There is no learning process. In this sense hydrology differs from some of the other sciences. By falsification we learn what we did not have in our original framework and original assumptions. Diagnostics can be a basis to separate the new from the old without getting into the semantics and all this.

Denis Hughes

Can I comment on the diagnostic methods. I have listened to a lot of papers over the last three days and in some of those papers I seen a lot of numbers and have never seen a hydrograph being shown. You need mathematical methods to do diagnostics, but these are not actually lines on a piece of graph paper or a computer screen. They are actually a hydrological response and while we are developing diagnostic methods please not forget that we are actually hydrologists and not mathematicians.

John Schaake

We now appear to have agreement on this broad strategic approach of five topics. In our publications we can have some of these views included and how this fits together.

Things to do within MOPEX:

" Include the MOPEX-2 Tucson Workshop results (data, model output, statistics, etc) on the MOPEX Website
" We need more models to participate in the science questions
" Upgrade the data access
" Data sets - Things we could do to help the ftp site
- National Weather Service site is being upgraded???
- Have the 438? basins in the US….

(Note these were listed from John's presentation and as the mike was not used all the time and can't confirm. John to check his computer presentation.)

Siva Sivapalan

We have quite a lot of clean data sets in Western Australia which could be provided.

John Schaake

" GEWEX data sets -

There are a lot of data global sets produced by GEWEX which we could use in our work. We could help you to understand what is there and provide access to some subset of these. For example, radiation, global reanalysis , a lot of the things that it takes to estimate evapotranspiration such as temperature. There are some data sets which we could make available. These would also be helpful to PUB. These could be made available through MOPEX for now and as PUB develops it could also provide assess. We need to list what is available.

Siva Sivapalan

One of the things we are asking the PUB working groups is if it is possible to make their data available widely so we can create a huge repository of MOPEX, PUB, FRIEND, HELP, basins, etc

George Leavesley

This does not have to be in one central repository. As long as we can get links to a variety of sites on the MOPEX page we can have hot links to wherever the data resides. So we can probably facilitate not having to buy big chunks of central storage in one place if everyone can support that.

We need to make it easier for other groups to contribute.

John Schaake

In summary we need lots of data sets to make this work. So getting this together and trying to get the information from various places in the world is the first step and then we need to determine how we are going to use it. We have some things in the mill with contributions from Germany and Austria.

Publications: as per John's/Yuan's presentation re timing, etc

It was agreed that two avenues of publication would be sought, the Journal of Hydrology for a possible special issue if sufficient papers, or the Hydrological Sciences Journal and an IAHS Red Book which would contain summary articles covering any Journal papers together with those of a less theoretical bent and covering techniques and model application. 250 word abstracts will be requested by email from the Workshop participants and some additional potential authors with submissions by the end of August???. Papers will be requested by the beginning of 2004 followed by three months for the first review. With another two months for revision and final review would allow publication by the end of next year.

John Schaake

We can say a lot in this Red Book about the science questions and what we are trying to do that might not go so well in the Journal of Hydrology. There are some specific results which we could show in a publication that is not limited on the graphs. We can control this so that some of the reports on what we have done could complement what is in the Journal. By publishing a Red Book we can help ourselves to cover the topic well and IAHS. We will give you permission for more than one paper.

Yun Duan

The Journal of Hydrology has a strict page limit of 15 pages. A Red Book would give us more flexibility.

Alan Hall

Cate Gardner has complained that some of the Red Books are just a collection of papers without an introduction or overview of the subject. Hence we should take out some of the material in the Journal and make it a self-contained and valuable publication.

Siva Sivapalan

I did not attend all the talks, but my understanding of the process that MOPEX is going through in terms of understanding how models operate, the diagnostics and the intercomparisons and so on. I missed any talks which showed that that kind of process has let to improvements in model structure. I would like to see, especially in the Journal articles you are have that the authors can be challenged to bring out some positive messages that this process ahs led to improvements or cross-fertilisation of ideas or concepts of one models versus another. That would be a very positive statement to make, not just to present the diagnostic results but using that to make an improvement.

John Schaake

That sounds great. I'll offer Yun's services as a guest editor and if anyone else would like to compete for that position we would certainly welcome your input on it. We'll get it done and if you do not respond we will come after you and you will get an email.

Future Workshops

George Leavesley

We have two Workshops to think about. One is in two years time in Brazil and we have little time to think about how to structure that. The idea of a Workshop 12 months from now to really explore some of the things just talked about. We have had a lot of interesting presentations over the last few days but we have really not had the opportunity time to get into the details, to look at how different people conceptualise different processes and then what methods do the use to make these a priori estimates. We need the time to sit down together and explore those issues. We would do that next year and the group at Cemegraf has been kind enough to offer their facilities as an option of holding a meeting there in France. The idea being that we would select perhaps half a dozen basins in different climatic and physiographic regions with a range of hard to soft data so that we could explore:
" how we make these a priori estimations,
" how do we integrate the soft and medium data sources,
" what are the best approaches on that and to be able to explore those issues, and

To this we need to have a variety of analytical tools, the tools that tease out some of the information on these parameters and the information that exists using measurable basin or climatic characteristics. We are not just thinking of only basin characteristics, we also need to look at the driving variables of these models as well and how to best estimate the distributions of precipitation, temperature, radiation and whatever we might be using in these various models. We offer this as a proposal and we would like to have some feedback on that.

Yun Duan

Last time we had the Tucson Workshop and we decided to have 12 basins. The way we did it may not be the only way and we welcome suggestions in how we do it. George has made an excellent start and we welcome further input from the audience.

Vazken Andreassian, Cemagref

My suggestion is to take more than a dozen basins. We got a lot of information from the Tucson 12 basins but now we need to shift to another scale. We should try to do something on 100 or 200 basins.

Yun Duan

We chose 12 basins from a practical point of view, taking into account the time available for people to analyse and give us the results.

Vazken Andreassian

From my point of view, getting the first basin set up with the reading routines, the data formats, etc takes the time. Once this is done 12 or 200 basins is the same. Maybe more complicated models or if you need manual calibration of course it is not to be done. For models which need only automatic calibration one model or 200 models take roughly the same time.

George Leavesley

We are also thinking of having the opportunity of exploring some of these models in real time on site so we would be definitely looking at a subset of that 100 to be able to address those issues, particularly soft data we might want to focus on a subset.

John Schaake .

We probably can't resolve this now. Maybe we can do all of this, in other words a few people might look at all of the basins and have some focus for what that might be. George has some great ideas on how think more cleverly about the relationship of what is going in the models and what is happening physically. We need to combine the top down and bottom up ideas in this. So having a focus on a few basins is necessary. We need more than one or two before we can conclude anything. Also we are not done with the first 12, we need some more models on these and it might be we use some of them as well. But the emphasis as we are thinking now will be to use some new ideas having to do with the soft data or the processes. What do we think we do with them are they doing what we think and some other things like that on a few basins and maybe with some additional data that we could use. Then if we could develop your idea Vazken, lets look at a lot of basins and a few of us might do that with some of the models. In fact if you could give us your model in the right form we could actually run some of that for you. We have some automatic things that could do something at some level and then we could look at what happened.

George Leavesley

With that and some sort of a ad hoc working group of myself, Hoshin Gupta, Thorsten Wagener, and …

John Schaake

It's the same Mafia that worked on this one with some extra help and since we are going to be at Cemagref, we'll add Vasken. It's an open group and anyone else that would like to join us would be most welcome.

George Leavesley

We will try and put a strawman together and circulate it for comment.

John Schaake

George has agreed to lead this one and Alan the Workshop in Brazil in two years so we might want to think about that one as well. We have a lot of issues there to think through so we are going to have to come back to for advice on it.

Alan Hall

On the first Workshop in Europe we want to cover other climate areas and other places and go much wider than just the US in terms of the basins. As for the Brazil Workshop we want to run this in conjunction with PUB and some of the other Commissions. But the idea there is to perhaps spend half a day to bring to the participants in Brazil the results of our Workshop at Cemagref. To some extent we have done that in the last three days. Within the programme in Brazil we cannot afford to spend three days. We would have half a day on key issue papers which would be open and invited. I am trying to convince the Programme Committee to have one half day of the five days set aside for posters. We can welcome a whole stream of posters and often a poster is the best way to present information on some of these topics. So we are looking at a Workshop which would run over 1 1/2 to 2 days.

John Schaake

I think we have to value posters as part of our process. If we are going to cut the time of our meetings down to one week instead of as it was two, we are going to have to use posters, or else we are going to lose contact, or else focus in very few topics, that's all.

Alan Hall

Part of this first Workshop is to get some good material , good ideas and information that we can feed back to the participants in Brazil.

John Schaake

The next topic I had was the community access, which was really the question of getting access to the models through the networks and so forth. We don't want to talk about it anymore, it's an idea we are pursuing. We have some suggestions as to how we can go about it and we would welcome your advice and participation. If you want to help us in some way please let Hoshin know if you have some ideas on how to do this and he will take the lead on this with us.

Hoshin Gupta

I'll make the facility available. I do not want to do it myself.

John Schaake

Let me or Yun know and we will find a way to get it done. I am no longer a Government employee, I am a contractor. Just give me the money and I willset the site up and run it.

Hoshin Gupta

My understanding of what we actually have in Tucson you do not need anything new. As part of SAHRA we developed a web access to a workgroup facility which allows a person to go in and submit files and download files, make comments on things, share presentations, all kinds of stuff. What you need is somebody to manage it, I think that would be you guys. The software is already there we just have to make people aware of how to get to it and we could put a link to it on the MOPEX site.

John Schaake

The final topic we had on our list was for us to reflect the way we are overseeing the MOPEX activities and to get any thoughts you have. This is not a top down bureaucratic operation so we have a real soft, loose oversight. Basically in my mind what it boild down to is that a few of us pay attention on a more routine basis and then these convenor groups that make these Workshops come together become an effective leadership group for the project. We don't want it to be closed. It is open, we once had a Steering Group but all we had was a list of people and we did not do anything. We now have a group of people who are doing something, let's call them oversight or management group.

Alan Hall

The way I see the way we have run it so far it is that each time we have a Workshop and get together at an IAHS Assembly we have a discussion on what are we going to do in the next couple of years. This seems to be reasonably effective planning ahead on a two year basis. It is an open process, we very much welcome input during that time and this is happening.

John Schaake

I would also encourage any of you folks that want to talk about your perspective on what MOPEX is all about and what MOPEX means to you to have a MOPEX presentation. It's ours, it's all ours. It is not mine or anyones. So if you want to use that I would like you to go speak about this go do it.

Hoshin Gupta

If you need more motivation, particularly to get more young people involved, it looks good on your resume.

John Schaake

Don't forget the five science questions we asked you to submit and the statements on MOPEX and the publications. I do thank you. I thank everyone who came, everyone who put there presentations together. This has been a fantastic meting, truly a nice meeting and I thank you for making it be that.