Introduction:
Far
West Laboratories has issued a request for proposal (RFP) to summatively assess the value of the Determining
Instructional Purposes package in order to recommend whether it is
worthwhile to invest in marketing the package of training units to a larger
audience or whether distribution should be abandoned completely.
The
evaluation is to be conducted between May 1 and December 31 of 2009.
Description of Program Being Evaluated:
The Determining
Instructional Purposes training package developed by the Far West
Laboratory for Education Research and Development was designed for increasing
the administration skills of school administrators and graduate students in
educational administration in order to plan more effective school programs.
The
units within the package are: Setting Goals, Analyzing Problems, and Deriving Objectives. The units may be administered in a
mix-and-match ordering style as deemed necessary by the purchasers of the
package..
Assessments
for the units have not yet been developed for the units, and an evaluation of
the package has not yet been undertaken.
Evaluation Method
i.
Purpose:
Summative
Assessment
ii.
Decision:
Determine
whether the three unit package:
Setting Goals/ Analyzing Problems/ Deriving Objectives should be marketed widely or
abandoned completely.
iii. Audience:
The
audience for the evaluation will be the staff at Far West Laboratories.
iv. Information Levels:
The
information collected will be qualitative interviews and surveys administered
to individuals at each level of the programÕs development and
implementation. Additionally, a
quantitative from assessments will be created.
v.
Data Sources & Instruments:
Source 1: Far West Laboratories Design Staff
Instruments: Interview designers to deduce strengths and weaknesses of the units.
Goal: Learn what instructional gaps need to be filled and the estimated work needed to do so.
Source 2: School districts (approximately 20) that have purchased the units.
Instruments: Survey school district superintendents in charge of purchasing in order to query the product satisfaction by those who have implemented the units.
Ideally, all 3 units will have been administered.
Goal 1: Identify level of satisfaction with the training materials.
Goal 2: Determine whether administrative performance has improved based upon the pre- and post- formal evaluations of administrative employees who had been trained within the district.
Source 3: Coordinators of the trainings.
Instruments: Interview coordinators to determine strengths and weaknesses of the units.
Coordinators should be the most valuable resource of the evaluation, as they have been both a participant of the units and an implementer.
Goal: Learn whether coordinators believe the units are successful in improving administrative skills as they are currently constructed.
Source 4: Participants of trainings (Used in districts implementing trainings during summer, 2009)
Instruments: Survey of participants to determine their feelings regarding their improvement and whether the program is worthwhile.
As yet undeveloped quantitative assessment of participant improvement.
Goal: Determine whether participants have actually improved their skills.
Task Schedule:
|
Task: |
Deadline: |
|
1. Meet with FWL to discuss proposal outline
and address any modifications. |
Friday May 1, 2009 |
|
2. Submit data-collection plan for all
instruments and draft copies of attitude surveys, interview
questions/protocol, formal evaluation comparison form, and quantitative
assessment. |
Friday May 15, 2009 |
|
3. Meet with FWL to receive feedback on
data-collection plan, attitude surveys, interview questions/protocol, formal
evaluation comparison form, and quantitative assessment. |
Friday May 29, 2009 |
|
4. Revise data collection plan and
instruments as appropriate.
Submit final copies to FWL. |
Friday June 5, 2009 |
|
5. Collect all surveys. |
Friday June 26, 2009 |
|
6. Collect all interview data. |
Friday July 17, 2009 |
|
7. Meet with FWL to present a summary of
survey and interview data. |
Friday August 14, 2009 |
|
8. Collect all formal evaluation
comparisons. |
Friday September 11, 2009 |
|
9. Collect all quantitative assessments. |
Friday October 9, 2009 |
|
10. Meet with FWL to present a summary of
formal evaluation and quantitative assessment data. |
Friday November 13, 2009 |
|
11.
Write final report and submit to FWL. |
Thursday December 31, 2009 |
Project Personnel
Craig
Miller, the senior evaluation coordinator at ES & A will
direct the evaluation and will have primary responsibility for planning and
implementing the overall evaluation, as well as interpreting the assessment
data.
Mr.
Miller is an experienced educator of 15 years who has worked with varying
administrations at large inner-city public high schools, private international
schools, and continuation schools.
He
will have his masters in Educational Technology through Boise State University,
Idaho in the Spring of 2009, and is experienced in
project creation and management.
Jeremy Tutty, Ph.D, the assessment design specialist at ES & A
will design the survey, interview, formal evaluation comparison, and
quantitative assessments. He will
also lead the data collection team and summarization of the data.
Mr.
Tutty is currently a professor at Boise State
University, Idaho. He has multiple
publications to his credit, and currently has two publications under review.
Lisa Dawley, Ph.D, the data collection specialist, will implement
the surveys, interviews, formal evaluation comparisons, and quantitative
assessments at the direction of Mr. Tutty.
Ms.
Dawly is currently the Department Chair at Boise
State University, Idaho, and a proponent of virtual worlds in education.
|
Personnel
Salaries: |
|
|
|
Craig
Miller: 30 days @ $350/day |
$10,500 |
|
|
Jeremy
Tutty: 30 days @ $300/day |
$9000 |
|
|
Lisa
Dawley:
45 days @ $200/day |
$9000 |
|
|
|
Total Personal:
$28,500
|
$28,500 |
|
|
|
|
|
Travel
and per Diem: |
|
|
|
Mileage:
1000 mi. @ $.30/mi. |
$300 |
|
|
|
Total Travel:
$300 |
$300 |
|
|
|
|
|
Communications: |
|
|
|
Telephone:
$150/mo. X 8 mo. |
$1200 |
|
|
Postage |
$250 |
|
|
|
Total Comm.:
$1450
|
$1450 |
|
|
|
|
|
Supplies,
Materials and Photocopying: |
|
|
|
Supplies
& Photocopying |
$600 |
|
|
5
Sets of Complete Units*
|
N/A |
|
|
5
Coordinator Handbooks*
|
N/A |
|
|
|
Total Supplies:
$600 |
$600 |
|
|
|
Total Budget: $30,850 |
*Copies
of units and handbooks may be needed if no districts who
have purchased the units are planning to train during 2009. In this case, free units will need to
be offered to ÒtestÓ districts in order to collect sample data.
Note: Educational Strategies & Associates
has a working relationship with many neighboring districts. The use of nearby districts will
drastically reduce travel costs.