Discussion #2 – 3/23/11

1.    Welcome and What’s Happening

2.    Forum Content

a.    Presentation and discussion topics

i.    We have sent you a list of presentation and discussion topics that combines ideas from last year’s Forum and proposals received this year.

ii.    What on this list is most important, lacking, unnecessary or surprising? Why? How could individual topics be improved and refined?

b.    Categories of content

i.    During the last call, the group began discussing the idea of creating 4 different categories (sub-themes?) for Forum discussions and presentations – conceptualization, design, management, and measurement and evaluation.

ii.    What are your thoughts on this approach? How would it work? How can we better structure communications prior to the event and networking at the event to accommodate this approach?

3.    Forum Format

a.    Skeleton agenda

i.    We would like general and specific ideas about improving the skeleton agenda. Please consider how the agenda and specific sessions might be best structured to achieve goals of learning and networking.

ii.    Is there appropriate emphasis on plenary sessions? Are they appropriately placed? What is the best format for a plenary session?

iii.    Is there appropriate emphasis on featured/keynote speakers? Are they appropriately placed on the agenda? What is the best format for these sessions?

4.    A Few Final

a.    Proposed trainings
b.    EEN products and services
c.    Facilitations
d.    Submit your proposals!

5.    Closing Comments

a.    Matt Keene kicked off the call with a summary of information distributed to date (including a feedback memo and summary of the 2010 event, a list of potential topics, and a skeleton agenda) and the agenda for the conference call.  Matt also provided general EEN updates:

i.    We would like to rotate featuring different organizations on the EEN website and would also appreciate feedback on the site. We will be posting some additional information about the complexity theme.

ii.    We have started a Twitter feed with posts each day, and have featured some of your organizations in them.

iii.    The call for proposals has been announced based on the theme of Navigating Complexity; we now have at least 20-25 suggested sessions and expect several more in the next few days.

iv.    Shelley Metzenbaum has invited Matt and Katherine to discuss EEN and ways of making it as successful as possible.

a.    Presentation and discussion topics: Prior to the call, we sent out a list of presentation and discussion topics that combines ideas from last year’s Forum and proposals received this year. What on this list is most important, lacking, unnecessary or surprising? Why? How could individual topics be improved and refined?

i.    Richard M: We should let the agenda come together organically once we view the proposals for sessions.

ii.    Christina / Steve: Agreed, it may be too premature to structure the agenda any further until we receive all proposals.

iii.    Mya: Novices vs. more experienced practitioners are looking for different things. Sessions directed to “Evaluation 101” could concur with different sessions targeted to experienced practitioners.

iv.    Alexandra: There is a divide between novices and more experienced practitioners – maybe name tags can include an indication of what stage they are in. Can we follow up with people who proposed topics after the 2010 Forum to find out more about what they were looking for?

v.    Ana: Keep in mind to balance the focus on public vs. private sector. Also, balance the focus on conservation vs. other environmental challenges/issues.

vi.    Katherine: Public sector organizations are sponsoring the event so it is difficult to make the case for shifting the focus away from them.

vii.    LaVanna: In marketing the EEN within my Bureau and Department, some employees saw EEN and the Forum as a think-tank for evaluators. To change that perception, I’ve tried to communicate the purpose of EEN and share with them how their involvement would be valuable for both the evaluation and the technical side of the environmental communities. As the Audit Liaison Officer for my Bureau, I also expressed how the Forum would enhance our internal communication and understanding across the disciplines when we’re conducting our internal reviews. This would ultimately aid in improving accountability in our programs by addressing our risks, challenges, and successes when the external reviews are conducted by the Office of Inspector and the Government Accountability Office. I would encourage each of you to advocate and market EEN using the same approach for a broader awareness within your respective agencies.

b.    Categories of content: During the last call, the group began discussing the idea of creating 4 different categories (sub-themes?) for Forum discussions and presentations – conceptualization, design, management, and measurement and evaluation. What are your thoughts on this approach? How would it work? How can we better structure communications prior to the event and networking at the event to accommodate this approach?

i.    Alexandra: We need more clarification on each of the four sub-themes.

ii.    Richard G: We could easily package breakout sessions into these four large buckets, while still having room to address the networking goals.

a.    We would like general and specific ideas about improving the skeleton agenda. Please consider how the agenda and specific sessions might be best structured to achieve goals of learning and networking.

i.    Richard G: Consider building a group activity into both days that brings people together at different junctures. Perhaps we build a representation of: 1) universe of environmental policies/programs (resulting in a typology of environmental evaluation), and 2) types of complexity that bear on environmental policies and programs. In advance, we could come up with a proposed portrayal of evaluation typologies; then provide a central workstation where people stop by to contribute/adjust.  A final facilitated activity could be to consider the intersection of environmental evaluation typology with the complexity typology to summarize what is at play in “Navigating Complexity in Environmental Evaluation.”

ii.    Steve: I like the idea of an activity that emphasizes complexity as an underlying component of the entire event. Maybe we can ask attendees to map their knowledge and experience throughout the Forum and possibly turn in the knowledge map at the end for synthesis. The complexity of how people think about their relationship to evaluation or how they use evaluation is interesting.

iii.    Alexandra: I like the idea of being able to tie sessions together, developing typologies or competencies that meet the needs of our practice. We could do a scavenger hunt of activities to get people involved – e.g., Tweet, post a LinkedIn discussion, meet a specific presenter.

iv.    Jonny: I can see value in the scavenger hunt idea as a way to deliberately enrich networking. With regards to the concept of complexity, it is very important in driving evaluation; it helps you think about your programs and might affect your logic models, methodology, etc. People should overlay thinking about their programs with how these concepts work.

v.    Mary: I like the ideas of laying out typologies and creating interdisciplinary connections.

vi.    Matt: Some of the ideas would result in drastic changes to the format of the Forum, whereas the existing format has received positive feedback. We can always improve the format, but how much we change it is what we must decide.

vii.    Richard M: We need to focus first on the “what” (topics proposed in submissions) and then later on the “how” (mechanisms and formats for hosting the “what”). Is there an opportunity for training either before or after the Forum, perhaps an all-day training?

viii.    Mya: The proposed agenda framework makes sense.

ix.    Ana: Agreed, the proposed agenda framework makes sense. Last year it was not clear how some of the sessions linked to the overall theme of the Forum. Too often, people were talking about their individual projects, but this did not always lead to a larger discussion or interaction.

x.    Christina: I appreciated the comments on the 2010 Forum on wanting more interaction. There still may be an opportunity to foster greater interaction through the sessions that are proposed.

xi.    Matt: We welcome ideas for how to make concurrent sessions more interactive.

xii.    Richard G: If it is not built into the agenda formally, networking and discussion is less likely to occur.

xiii.    Katherine: Some people attend conferences just to listen and learn from other people and are not going to participate in the networking portions.

xiv.    Alexandra: When publishing the agenda, we should point out what we have tweaked so people can see that we are open to ideas. We should also include different ways of searching for sessions, e.g. by format, topic, etc. This could be as simple as an index in the printed materials.  Could we model this search option on the Transportation Research Board (TRB) Annual Meeting website (http://www.trb.org/AnnualMeeting2011/Public/AnnualMeeting2011.aspx)? If not, does the current EEN website allow us to develop document libraries that are searchable the way you would on a SharePoint (e.g. tag documents with certain attributes such as file type, title, and create views based on those attributes/and/or facilitate searches based on those attributes?) I believe that EEN website visitors could then print their search results and be able to make this their customized agenda.  I just think that there must be a way to do this using the existing EEN contracts/resources.

xv.    Katherine: Searchable electronic agendas tend to be expensive formats to use. We are open to ideas for less expensive methods.

b.    Is there appropriate emphasis on plenary sessions? Are they appropriately placed? What is the best format for a plenary session?

i.    Richard G: Anchor people on the life-cycle concept when designing plenary sessions.

c.    Is there appropriate emphasis on featured/keynote speakers? Are they appropriately placed on the agenda? What is the best format for these sessions?

i.    Matt: To identify the keynote speakers, we usually try to engage a leader of AEA, as well as political appointees with an interest in the field. We welcome the Advisory Group to recommend keynote speakers.

ii.    Katherine: We always invite our own senior management and that of other co-sponsors. We also tend to target some individuals to speak from an international perspective.

a.    Proposed trainings

i.    All-day or half-day training has been proposed several times.

ii.    Jonny: Logic model training is a good place to start.

iii.    Ana: Evaluation 101 training would be valuable, as well as training on GIS and other tools.

iv.    Richard: 101 training could work as a precursor to the event.

v.    Mya: In the registration form, we could try to get a sense of whether the community would be interested in 101 training.

b.    Advisory Group members are encouraged to think about product and service ideas that might come out of the event.

c.    If you have ideas for facilitation, please let us know.

d.    Submit your proposals!