Tuesday, April 27, 2010

Draft expert opinion Survey

In stage 1 of the expert opinion gathering, the part where we calibrate the state-and-transition model and estimate how well supported it is, this is what I would envision presenting to repondents.

In some cases there might be ecosite-specific questions to help resolve places where I am unsure, e.g. "Do you believe that S1 and S3 are distinct states?" or something like that, but this would be the general template of questions.

*** = required questions

I think that a person might spend less than 10 minutes on the required questions, but could spend up to 20 minutes more on the more open-ended questions.

what do you think?? any additions needed. I think number 8 is important, but is it obnoxious?



***1)Please identify any states or phases which should be omitted from the state-and-transition model.

Multiple choice, e.g…..
a. P1
b. P2
c. S1
d. S1P1
e. S1P2
f. S2
g. S3
h. None, none all should be retained

2)Please identify any states or phases which are currently omitted, but should be added to the state and transition model. Please briefly list dominant species, dominant processes and key characteristics, and appropriate references if they are available.

***3)Please identify any transitions which should be omitted from the state-and-transition model.

Multiple choice, e.g….
a. T1
b. T2
c. T3
d. T4
e. T5
f. T6
g. T7
h. T8
i. T9

4)Please identify any transitions which are currently omitted but should be added to the state and transition model. For each addition provide, the starting state and ending state for which the transition applies. Also please provide a brief explanation of the process that brings about the transition, e.g. fire, insect outbreak, drought, grazing.

5) Please identify other changes you think should be made to the draft model, if any.


***6)Please estimate your overall confidence that a new model which takes into account your proposed modifications is the correct model of the most important ecosystem states, processes and dynamics of the ecosite in question.

[Please answer on a subjective scale of 0 – 100% certainty. Enter any value in this range. To help you answer: 0% means “It’s anyone’s guess, this model is no better than any other model”, 50% means “Because this model is reasonable I would tend to believe it until evidence to the contrary is presented”, 100% means “The model is so well-supported by evidence and accumulated knowledge, that I am certain it is correct.”]

7)If your level of confidence in any particular state or transition differs from the value above please estimate your confidence for that model component in the appropriate box. In case you are estimating a confidence in a state or transition suggested by you in questions 2 and 4, please use the “other” boxes to identify it.

If you do not provide answers to 7 we will assume they are the same as the answers to 6 in all cases.

****8) Please take a moment to think of any scientist or other person, who is to your knowledge the best qualified to develop a state-and-transition model for this ecosite. This person could be yourself, or any other person. “Best” qualified may or may not mean highly qualified. Now, in the hypothetical scenario that this person had prepared a state-and-transition model for this ecosite using all of the data, knowledge and experience available to them, estimate how much confidence you would have that it is the correct model of the most important ecosystem states, processes and dynamics of the ecosite in question.

[Please answer on a subjective scale of 0 – 100% certainty. Enter any value in this range. To help you answer: 0% means “It’s anyone’s guess, any person could produce an equally good or bad model”, 50% means “Because this model is reasonable I would tend to believe it until evidence to the contrary is presented”, 100% means “The model is so well-supported by evidence and accumulated knowledge, that I am certain it is correct.]

2 comments:

  1. 1. Re your question 8, I don't think it's obnoxious. But is it your intent that the respondant identify the perceived expert here? This doesn't seem to be an explicit part of the question, but perhaps should be.

    2. It seems that we should also prompt and provide opportunity for respondant to comment or elaborate on individual states and/or transitions depicted in the model - i.e., even if the overall structure of the model is judged to be adequate. For example, I might think that the model looks good overall, but I might suggest some modifications to the descriptions of the states and transitions. Specifically, we could prompt for input regarding spatial or temporal 'contingencies' that may increase or decrease the likelihood of a particular transition. Similarly, we could prompt for input regarding 'at risk phases' (sensu Briske et al. 2008) within states. I suggest reviewing Steve's threshold attribute table, which is designed to trigger thinking about various threshold components / processes, and whether there is a way to simply prompt for such input using your survey format.

    3. Should we provide some introductory / background material that briefly describes some key assumptions re STM framework? There may be people who are very knowledgeable about a particular system, but who may not explicitly conceive of system dynamics in an STM framework - or who may misunderstand some aspects of the STM approach.

    ReplyDelete
  2. 1. okay we could make that an optional answer. people might feel a little wierd stating that they are the single most qualified person, so we shouldn't force them into it. but it could help identify a contact that we are missing.

    2. I think that was the purpose of question number 5, though perhaps we should prompt respondents in a more specific way rtaher than leave it so open ended. I'll work on this. There will be a more detailed stage 2, where specific questions are asked about specific transitions, where some of this might fit better (need to draft this still).

    3. Good idea. I want to avoid making this tedious (since i'll probably want to hit up people for more than one survey), but a paragraph length explanation is certainly in order. Will do.

    i'll nag steve for comments then post a draft 2.

    ReplyDelete