How will you know that your broader impacts program and its respective activities are actually achieving the desired outcome? Regardless of whether you are thinking about executing broader impacts in general, or specifically for NSF, evaluation is a critical component to determine BI success.
NSF expects that you will carry out the activities described in your proposal. This includes evaluating the effectiveness of your broader impacts activities. NSF states that meaningful assessment and evaluation of NSF funded projects should be based on appropriate metrics, keeping in mind the likely correlation between the effect of broader impacts and the resources provided to implement projects. If the size of the activity is limited, evaluation of that activity in isolation is not likely to be meaningful. Thus, assessing the effectiveness of these activities may best be done at a higher, more aggregated, level than the individual project (18 -1). This means also evaluating at the program level.
New Note: Furthermore, even if assessment of broader impacts outcomes for particular projects is done at an aggregated level, PI's are expected to be accountable for carrying out the activities described in the funded project. Thus, individual projects should include clearly stated goals, specific descriptions of the activities that the PI intends to do, and a plan in place to document the outputs of those activities (NSF 18 - 1).
Evaluation is a necessary part of Broader Impacts and BIR can help with small and large programs and projects!
BIR has partnered with ETEAM: http://eteam.ou.edu/. The Educational Testing, Evaluation, Assessment, and Measurement (E-TEAM) designs research tools and evaluations to help organizations understand and use data to solve real-world problems. E-TEAM works on projects for the public and private sectors that have a lasting effect on the community. The program helps community leaders determine what is best for the community through evaluation, assessment, and research. E-TEAM is the only evaluation organization in Norman Oklahoma that is a member of the American Evaluation Association (AEA).
For more information on ETEAM email Geneva Strech: firstname.lastname@example.org
Here are some evaluation tool kits to understand and evaluate your broader impacts program and activities.
Evaluation Tool Kit I
Why do I need an outside evaluator: http://broaderimpacts.missouri.edu/documents/evaluation/Evaluation_Outside_Evaluator.pdf
Evaluation Tools and Methods: http://broaderimpacts.missouri.edu/documents/evaluation/Evaluation_Tools_and_Methods.pdf
Evaluation Resources: http://broaderimpacts.missouri.edu/documents/evaluation/Evaluation_Resources.pdf
Program Assessment Toolkit II
- A guide to conducting interviews and surveys: http://www.wcer.wisc.edu/publications/leadcenter/toolkit.pdf