Assessment Plan

The assessment of the student learning outcomes for the QEP will be accomplished by a combination of one nationally-known assessment instrument for critical thinking, several questions from the National Survey of Student Engagement (NSSE), the critical thinking subscale of the ETS Proficiency Profile and individual rubrics designed to assess both faculty-made assignments and student work samples.

National Assessment Instruments

The Critical Thinking Assessment Test (CAT) published by Tennessee Technological University was chosen as our primary external assessment instrument.  According to information from Tennessee Tech, the CAT is “a unique tool designed to assess and promote the improvement of critical thinking and real-world problem solving skills.”  The instrument was designed with the assistance of the National Science Foundation and that factor too entered into the decision.  

The choice of the CAT as the primary external assessment instrument for critical thinking has implications for budgets and faculty / staff resources.  The test will be scored by members of the Pfeiffer community, primarily faculty members, and this will require the assessors to receiving training in scoring the test.  This feature is an important link in the “closing the loop” process and can be seen in the following graphic.

The figure below shows how the CAT is designed to address the major elements of Bloom’s Taxonomy of Cognitive Skills (1956).

ETS Proficiency Profile
Pfeiffer has for years used the Measure of Academic Proficiency and Progress (MAPP) as an instrument during senior exit testing, and although this instrument has now changed to the ETS Proficiency Profile, Pfeiffer seniors still take it.  One of the subscales of the Profile is critical thinking and in addition to the CAT, these subscale scores will continue to be monitored for changes once the QEP has been implemented.

The National Survey of Student Engagement
Pfeiffer has participated in the NSSE for many years and thus, unlike the CAT, a good bit of baseline data has been gathered.  A number of questions from the NSSE specifically address student engagement in academic life and these questions will be monitored in order to uncover any improvements in the degree of engagement of students with academics. (See Table 2 above for list of specific topics of relevance.)

Internal Measures and Rubrics

The institution’s faculty and students will have access to two generalized rubrics that provide descriptors of the elements that lead to critical thinking and engagement. For faculty, a rubric has been drafted that describes what an engaging assignment includes. The elements include the criteria of level of student engagement, reflective practice, the level of cognitive ability, the level of affective ability, and the level of student-directedness. Descriptors for meeting the standard, exceeding the standard, and not meeting the standard are provided. Faculty assignments that are purported to engage students and drive critical thinking, to be included in the exploratory projects and eventual full implementation of QEP, will be reviewed by the QEP director for appropriate level of standard. For student work samples, a rubric has been drafted that provides criteria and standards for the results of student effort. Again, the descriptors provide examples of the levels of work that meet standard, exceed standard, or do not meet standard. Faculty will use this generalized rubric as is, or derivations of it in primary trait analysis scales, to share with students prior to and following the assignments. Over time, as the rubrics are used again and again, students and faculty will both naturally internalize the contents of the rubrics to help focus on engagement that leads to improved critical thinking.

The assignment rubric will be used to judge the quality of the faculty assignment while the critical thinking rubric will be used to provide feedback about individual student work. These rubrics will be made available to faculty members and they will be trained on their use by the Director of the QEP. 

At the conclusion of each academic year, the collection of data points will be reviewed by the QEP director, the Office of Institutional Research, and the deans, to determine what progress has been made and to determine what faculty development in terms of assignment construction and course delivery might be appropriate. The Office of Institutional Research, which has responsibility for the administration of all external measures, will review data to ensure that response rates are appropriate for each instrument and that the method of administration provides the best response rates. The external data will be aggregated for inclusion in the annual institutional fact book and will be dis-aggregated to provide each faculty individual and each program collectively with a perspective of student progress and opinion. Collectively, faculty will review these data to determine appropriate changes to curriculum and delivery.