Rich Kids, Poor Kids Scenario:

The Maricopa Community College District (MCCD) partnered with Cashflow Technologies, Inc., to design and deliver a course in financial literacy. The course is based on the products - books, games, and videos - created and sold by the company. Complements to the course include training sessions for future course instructors and seminars at which attendees play the Cashflow game. MCCD commissioned an evaluation of the program to verify its value to students and the community. In addition, Cashflow Technologies is exploring the possibility of marketing the educational program to educational institutions throughout the nation; an educational evaluation of the program might lend it credibility. Cashflow Technologies has been producing the materials for three years. The course was offered for the first time during the spring 2008 semester. No training sessions or seminars have been performed.
The program will be evaluated by students in Educational Technology at Boise State University. The evaluators have access to sales information for the products, student performance data from the course, and contact information for the students and instructors. The budget for the evaluation is $6,000.

Evaluation Questions:
1) What are the benefits and limitations of an evaluation?
Benefits for future students:
• Evaluation may lead to an improvement in products and/or practices that will help future students attain a greater level of financial literacy than students who have previously than the course as well as those who do not take the course.

Benefits for community’s future:
• If evaluation leads to greater financial literacy in future students, the students who choose to settle within the Maricopa district will make better financial decisions in their lives, thereby providing a long-term “impact” upon the areas future economic soundness.

Benefits for MCCD:
• Discussion/collaboration among the MCCD staff in order to objectify the term “valuable”, making it a measurable term which may be assessed by the evaluation team.
• Learn whether the course design and delivery as currently constructed provides valuable immediate “outcomes” for students and the community (This is the stated purpose for the evaluation).

Benefits for Cashflow Technologies:
• Learn whether the Cashflow products (books, games, and videos) are valuable for students and community in their present state of development.

Limitations for MCCD:
• The course has only been taught for a single semester (or perhaps two semesters if it was offered during the Summer of 2008). This means there is a very small sample size from which to pull data. Most likely, the procedures of the course have not yet been reflected upon and revised for peak student performance. It should probably be suggested to MCCD prior to commissioning the evaluation that the course activity sequence be agreed upon as ideal and implemented for a few semesters in order to increase the sample size. (An increased sample size naturally balances statistical variation, leading to greater confidence that the data is accurate.)

Limitations for Cashflow Technologies:
• As the evaluation may be considered “high-stakes” for a company hoping to expand, Cashflow Technologies may confound the results by “framing” any information they provide so that the company appears in the most positive light possible.
• The evaluation is only taking place within the Maricopa District, which may have specific socio-economic characteristics that make the results inaccurate or too unique for using the data as a basis for comparison with areas possessing a collection of more typical demographics. (Another reason why a large and diversified sample size is preferable to a small sample set.)

2) What factors ensure that an evaluation will be successful?
• Evaluation is being performed by Educational Technology students at Boise State University, an independent and presumably objective group.
• Evaluation team has access to objective student performance data from those who have taken the course.
• Evaluation team also has contact information for both students and instructors in order to gain any necessary clarifications and/or collect subjective anecdotal evidence.
• $6,000 dollars should be enough to fully compensate the evaluation team so that they will not need to limit their efforts or require MCCD to select fewer services during the evaluation process.

3) How might one use evaluation results?
Potential uses for MCCD:
• Determine whether solely basing the course on the Cashflow books, games, and videos without the implementation of the intended training sessions for instructors and/or seminars for players of the Cashflow games is valuable for the students and community.
• Determine whether to continue using Cashflow products as currently constructed or to ask for their modification.
• Determine whether to suggest the development and addition of trainings/seminars for the course.

Potential uses for Cashflow Technolgies:
• Independent validation of products would give promotional support for the current product lines.
• Validation would give a rationale for expansion of products/services.

Potential uses for educational institutions outside the MCCD :
• Determine whether the Cashflow products have value when considering whether to purchase and implement them within their own institution.