EdTech 505 Home

Evaluation Scenario

Group Definition Page

Project Idea Page

Evaluation Proposal

Evaluation Plan Review

Evaluation: Response to A Solo Performance

Final Project Documents

Survey Data Report

Task List

Observation Protocol

Opt Out form

Instructional Evaluation Survey

 


Apendix C

Evaluation of Determining Instructional Purposes Courseware

A Proposal Submitted to
Far West Laboratory for Educational Research and Development

By
The Desmarais Research Group

 

Introduction:

Desmarais Research Group , in response to a September 2008 request for proposal from The Far West Laboratory for Educational Research and Development, has developed the following plan for evaluating the Determining Instructional Purposes training package.

Determining Instructional Purposes:

Developed by The Far West Laboratory for Educational Research, the Determining Instructional Purposes training package is a modular package of instruction designed for training school administrators and graduate students in educational administration to plan effective school programs.

The training package is divided into three units of instruction, each designed to be either a stand alone course, or part of the whole. Determining Instructional Purposes (DIP) is a print-only course that uses a Coordinator’s Handbook, and three modular bloacks of instruction. The Handbook costs $24.95 and the units are available as indivual purchases at $49.95 each, or as a whole for $129.95.

The courseware is designed to be truly modular in form, so the individual units might stand alone as single courses. The course offers the option of being presented as a workshop or as a series of shorter classes. The estimated training time required for the entire program is 32-48 hours. As individual modules of instruction, Units 1 & 3 require an estimated 10-15 hours each, and Unit 2 requires an estimated 12-18 hours for completion.

The training package is designed so all acitvities are directed by a program coordinator, who will organize guide, and monitor the participants as they work through the lessons.

Evaluation Methods:

The evaluation methods and data sources, as described below, will be tailored to the project in order to provide the most accurate result possible.

Materials Review: The staff at DRG will review the materials in the DIP packets in depth and provide a detailed analysis the courseware. This review will be aimed at the overall design of the instruction. Does it follow a best industry practices in terms of the actual design of the instruction itself? Design flaws will be measured against discovered strengths to guide DRG’s reccomendation to Far West on the future of this training package.

Implementation: A telephone survey of known purchasers of the training materials to determine the level of deployment of the DIP by those organizations who have purchased the training units. The level of deployment will play a major role in how the evaluation will be conducted.

Attitudes of Policy Makers: A telephone survey of those administrators who actually deployed the training packets for use in their school systems to determine if the quality of new programming in their schools had been improved after staff members completed the DIP course of instruction. Are they happy with the courses? Have new programs been propagated or old programs improved as a result of administrator’s use of the DIP instruction.

Observation: Direct observation of randomly selected sample groups of personnel participating in the training module, to determine the efficacy of the program coordinators.

Attitudes of Participants: Using an instrument to be designed by member of the DRG staff, survey participants attitudes toward the training. Specifially on their attitudes as to whether or not they feel prepared to produce high quality programs for their schools as a result of receiving this training.

Task Schedule:

A complete breakdown of the tasks planned for this evaluation can be found in Appendix A of this response to proposal. The schedule assumes a start date of May 1, 2009 and a final report due date of March 1, 2010.

Bidder Qualifications:

The Desmarais Research Group specializes in instructional design with a recognized expertise in developing and evaluating workshop and corporate training courseware. The company was founded in 2002. DRG designs and evaluates executive training courses for General Electric, The Adult Education Consortium, and Pitney Bowes. In addition, DRG has designed the professional development program for the public schools of the town of Bethel, Connecticut, and has an
ongoing consulting relationship with the University of Connecticut School of Education’s Instructional design program.

Project Personnel:

A complete list of project personnel with brief descriptions of responsibilities and qualifications can be found in Appendix B of this response to proposal.

Budget & Schedule of Fees

Complete details of the proposed budget along with the schedule of fees for this project are avaiable in Appendix C of the proposal.

Appendix A

Task Agency Responsible Deadline Date
Meet with staff at Far West Laboratories for Educational Research and Development to discuss DRG’s proposal and make any desired modifications to the proposed evaulation. DRG 5/01/09
Supply DRG Researchers with 4 copies of the complete training training packet to include Coordinators Handbook. Far West 5/01/09
Submit data collection plan for all intruments and draft copies of surveys, interview protocol as well as criteria for on site observations to Far West Laboratory for approval. DRG 5/16/09
Provide DRG with feedback on data collection plan, evaluation instruments and protocols Far West 5/30/09
Revise data collection instruments ad interview protocols and interview criteria if necessary to meet client approval DRG 6/14/09
Collect all interview, survey, and observation data DRG 8/31/09
Summarize interview, survey, and observation data . Meet with Far West staff to present project to date. DRG 10/31/09
Send data on student achievement (if possible) to DRG Far West 11/05/09
Summarize achievement data. Meet with Far West to present and discuss project to date. DRG 12/01/09
Write final report and submit to Far West. DRG 12/31/09


Appendix B

Dr. Francis W. Lonchamps, senior partner at Desmarais Research Group will coordinate the evaluation of Far West Laboratory for Educational Research and Development’s training package; Determining Instructional Purposes. Dr. Lonchamps has been principal evaluator at DRG since its founding, and is a recognized industry expert in workshop evaluations. He has written a number of well received scholarly papers on the subject of professional development courseware. Dr. Lonchamps hold a Ph.D. in Instructional Design from Harvard University.

Tina Marie Louise will serve as principal evaluator for the evaluation of Determining Instructional Purposes project. She has been a lead evaluator on seven major instructional evaluations. She has designed courseware for Fortune 500 corporations, public school systems, and connecticut’s Home Schooling Society. Ms. Louise earned an M. ET. From Boise State Univeristy, and a B.S. in statistics from M.I.T.

Benjamin Nestor, an instructional designer at DSG will be designing the instruments and data collection tools to be used during the evaluation of Determining Instructional Purposes for Far West Labs. He has worked on design projects with DSG for three years. Mr. Nestor holds a M.S. in Education Technology from San Diego State University.

In addition to the named principals of the evaluation team, DSG will assign specialists to various portions of the evaluation on an as needed basis. The evaluation team will also include a graduate student from the University of Connecticut.

Task Agency Responsible Deadline Date
Meet with staff at Far West Laboratories for Educational Research and Development to discuss DRG’s proposal and make any desired modifications to the proposed evaulation. DRG 5/01/09
Supply DRG Researchers with 4 copies of the complete training training packet to include Coordinators Handbook. Far West 5/01/09
Submit data collection plan for all intruments and draft copies of surveys, interview protocol as well as criteria for on site observations to Far West Laboratory for approval. DRG 5/16/09
Provide DRG with feedback on data collection plan, evaluation instruments and protocols Far West 5/30/09
Revise data collection instruments ad interview protocols and interview criteria if necessary to meet client approval DRG 6/14/09
Collect all interview, survey, and observation data DRG 8/31/09
Summarize interview, survey, and observation data . Meet with Far West staff to present project to date. DRG 10/31/09
Send data on student achievement (if possible) to DRG Far West 11/05/09
Summarize achievement data. Meet with Far West to present and discuss project to date. DRG 12/01/09
Write final report and submit to Far West. DRG 12/31/09