Philosophy Discipline Assessment 2006-2007

Scope of assessment activities

___⇒__Course-embedded assessment

___⇒___ Pre- and post-testing

______ Outside the classroom

______ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

______ Portfolio assessment

______ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

______ Qualitative internal and external juried review of

of comprehensive senior projects

______ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Philosophy discipline objectives. The philosophy program offers students the opportunity to

· study the works of significant figures in philosophy

· investigate the fundamental problems and systems of thought that frame philosophical inquiry

· develop the ability to think and write critically and effectively

· cultivate the logical, analytical, and conversational skills necessary for stimulating and fruitful philosophical inquiry.

2. Capstone course. Senior philosophical defense.

There are five primary goals for this course, which aim at ensuring that the student can

· give a clear, in depth written exposition of some view in philosophy

· defend or criticize some view in philosophy

· give a clear oral presentation of some view in philosophy

· give a clear oral defense of some view in philosophy

· conduct a thoughtful discussion with peers and faculty on some view in philosophy.

During the course philosophy majors develop a substantial piece of philosophical writing, produce multiple drafts in response to comments from the whole philosophy faculty, and then orally defend the thesis.

3. Evaluation and assessment.

The philosophy faculty’s practice has been to discuss “the achievement of the program’s and defense’s goals only orally at the end of the majors’ senior defenses.”[1] That is, the degree to which each philosophy major achieved specific discipline and defense goals has not been measured quantitatively in the past. This will change next year with the introduction of numerical assessment tools that the faculty will use for each student and each programmatic/defense goal. See Faculty Senior Philosophical Defense Questionnaire and the Faculty Philosophy Questionnaire included in the appendices.

4. Course-embedded assessment. Pre-test/post-test

Introductory ethics. This course has no prerequisites and would be the introduction to formal philosophical discourse for many of the forty-five students in the course. The instructor, in a context that had nothing to do with course grades, sought to measure students’ “ability to present and critically evaluate arguments from our texts.”[2] The texts used for pre- and post-test were philosophical essays of comparable difficulty. The students’ twofold task in each was to “present a valid numbered argument” about the essay and to “identify premises that were vulnerable to criticism.” These tasks, of course, are a subset of the discipline’s goals. The pre-test was given during the seventh week of the semester after students had been exposed to philosophical terms and methods. The post-test was given at the end of the semester. Only a small fraction of the students on either test successfully completed the task, but whereas in the pre-test almost everyone was far from success, almost everyone on the post-test was on the “right track.” The instructor reports that this assessment tool suffered from its “all-or-nothing” design—students were successful or not. In the future he will use a sliding scale, will use articles and opinion pieces from the popular press in addition to philosophical essays, and will similarly measure student progress in acquiring conversational skills.

5. Student evaluations of the program

The philosophy faculty has relied heavily on student opinions of the major for making programmatic changes. Examples of these student surveys are included in the appendices.

General education categories spanned by the discipline

Each philosophy course carries one of the following general education designators: Hum, communication, language, literature, and philosophy; M/SR, mathematical/symbolic reasoning; E/CR, ethical and civic responsibility; SS, human behavior, social processes, and institutions; or Hist, historical perspectives. Exceptions are directed study and the senior philosophical defense, which carry no general education designator.

Physics Discipline Assessment 2006-2007

Scope of assessment activities

___⇒__Course-embedded assessment

___⇒___ Pre- and post-testing

______ Outside the classroom

______ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

______ Portfolio assessment

______ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

______ Qualitative internal and external juried review of

of comprehensive senior projects

______ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Physics discipline learning objectives. Students will

· acquire an understanding of the concepts of classical and modern physics

· learn to formulate and solve quantitative problems

· acquire the ability to experimentally investigate physical phenomena

· learn to communicate, in form and content, both verbally and in writing, the results of scientific work.

2. Capstone experience: Senior Thesis.

The goals of this course reflect discipline learning objectives:

· Familiarization with current research topics in physics.

· Familiarization with how to search and read physics research literature.

· Develop skills of expository scientific writing.

· Develop skills of oral presentation of scientific ideas.

· To apply undergraduate knowledge in physics to current research topics.

Through a collaborative process with other students and faculty, each student develops a proposal to study a topic of current research interest in the physics community. After approval of the topic, the student works with an assigned faculty advisor to develop the paper and oral presentation. The end products are a written report and an oral presentation. (Drafts and practice presentations are required.) The physics faculty meet to judge the extent to which each student came to understand the topic, how well the student utilized the research literature, and how well the student presented the topic in both written and oral presentations. The faculty also discuss how the course could be altered to increase students' achievement of the course objectives.

In the past year or two, the faculty observed a need for more structure in the course to keep students on track during the long periods of time allotted to study and writing. As of the fall of 2007, the faculty implemented a series of new milestones in the course in order to provide such structure. These milestones will encourage students to stay on track and will provide more opportunities for specific and detailed feedback from faculty as students are working on their papers and presentations.

A couple of years ago, the faculty observed that some students were having difficulty selecting topics for senior thesis projects. Due to a lack of awareness of current research, the students were taking too long to identity a topic, which resulted in less time to do the requisite study and writing, and contributed to poorer outcomes. In part to address this, the physics discipline created a course known as the "journal club," in which students interact with current research literature in physics in a more informal, discussion-based format. One goal for the journal club was to expose students to current research topics earlier in their undergraduate years so that they would have a better start on the first two objectives of senior thesis. Because this new course has only been in place for two years, this fall will provide the first opportunity to observe whether this makes a difference for students in Senior Thesis.

3. Course-embedded assessment. Pre-test/post-test

Stars, galaxies and cosmology.

This is a general education course without lab in the physical science category. The instructor uses a variation of the pre-test/post-test method with a variety of experiences in-between to enhance student learning. An example[3] is the distance modulus equation, which relates the apparent and intrinsic brightness of a star to its distance from the observer. The pre-test is a worksheet that the instructor collects, evaluates, and records a score for each student; the score provides both a benchmark for gauging improvement and an indicator of weaknesses and strengths in student performance. A quiz on the topic is used in the same way. Besides being used as assessment tools, the pre-test and exercises are graded for participation/effort. A problem on the midterm constitutes the post-test on this particular topic, gauges how successfully students mastered it, and provides one item for determining the course grade.

General education categories spanned by the discipline

Physics courses bear one of two general education designators, either Sci, physical and biological sciences without lab or Sci-L, physical and biological sciences with lab. Journal club, directed study and senior thesis carry no general education designator.

Political Science Discipline Assessment 2006-2007

Scope of assessment activities

______Course-embedded assessment

_______ Pre- and post-testing

______ Outside the classroom

______ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

______ Portfolio assessment

______ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

______ Qualitative internal and external juried review of

of comprehensive senior projects

______ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Three subfields

Students majoring in political science must choose one of three subfields in which to concentrate their studies: American politics; international relations and comparative politics; or political theory. There is a capstone course for each subfield.

2. Political Science discipline learning objectives are

· to be able to critically analyze, interpret and synthesize the major theories that are prevalent in a major subfield of political science

· to become more empowered to participate in government due to increased familiarity with politics and government

· to be adequately prepared for entrance into graduate or professional school.

3. Capstone courses for the three subfields

3.1. A fully implemented course.

The course became mandatory for majors with the 2003-2005 UMM catalog. However, students can graduate from UMM under the requirements of any catalog in effect during their enrollment, with the consequence that until the class of 2007, not all majors took the capstone course. However, the course is now fully implemented with all twenty-one majors who graduated in 2007 having taken it.

3.2. Capstone requirements.

Students write a scholarly paper and make an oral presentation of their results. They enroll in the course in the fall semester of their senior year. A minority of them finish in the fall, but most require both semesters to complete the requirements.

3.3. Assessment tool.

All faculty met to evaluate all papers and presentations with respect to eleven assessment criteria that reflect the three broad discipline goals. For each criterion, each student’s work is rated at one of three levels: failed to meet, met or exceeded expectations. “Expectation” refers to quality, that the quality was commensurate with a political science graduate at a top liberal arts college and would be suitable for presentation at a top undergraduate research conference. Among the criteria is whether the student demonstrates readiness for graduate, professional or law school.

3.4. Assessing student learning.[4]

The author of the discipline report notes many positive results in this year’s capstone students: 50 to 75 % of the students wrote well, showed good scholarly editing skills, gave good presentations, showed adequate knowledge of the field, and demonstrated an overall command of the material. On the downside, many of the papers would not be suitable for presentation at a top undergraduate research conference; many did not demonstrate proper methodological rigor, and had inadequate literature reviews as well as unclear or poorly developed hypotheses. The faculty judged that about half of the majors demonstrated readiness for graduate, professional or law school.

3.5. Improving student learning.

3.5.1. In the capstone course.

Now that the capstone course has been fully implemented, and the faculty has gotten a comprehensive and detailed look through the course at learning among its majors, a dozen changes have been recommended for the capstone course and its assessment.[5] Some of the proposed changes are more or less mechanical in nature, but some are substantive: in the future the faculty will measure information literacy and the student’s ability to adequately contextualize the “fit” of their paper within the field; and there will be a greater emphasis throughout the curriculum on the differences between scholarly and non-scholarly sources.

3.5.2. In the program.

Four changes have been proposed. One change, the need to offer International Relations Theory annually, is driven by the failure of several students in the capstone course to demonstrate an adequate theoretical understanding of the field. The faculty also recognizes the need for major changes in the political theory subfield because of the poor papers and presentations by the two political theory students in the capstone course.

3.6. Student surveys.

Tables 2-4 in the body of the discipline report give numerical summaries of student self-assessment of the degree to which they have met the three discipline learning objectives. Their written comments appear in Appendix B. Appendix A is the survey instrument itself.[6] The student surveys helped shape changes to the capstone course and the program.

General education categories spanned by the discipline

Political Science courses bear one of the following general education designators: E/CR, ethical and civic responsibility; SS, human behavior, social processes, and institutions; Hum, communication, language, literature, and philosophy; HDiv, human diversity; IP, international perspective; or Hist, historical perspectives. Exceptions are directed study and field study in political science, which carry no general education designators.

Psychology Discipline Assessment 2006-2007

Scope of assessment activities

__⇒___Course-embedded assessment

_______ Pre- and post-testing

______ Outside the classroom

__⇒___ Across the discipline

Direct measures of student learning

__⇒___ Capstone experience

______ Portfolio assessment

______ Standardized tests

______ Performance on national licensure, certification or

pre-professional exams

______ Qualitative internal and external juried review of

of comprehensive senior projects

______ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Psychology discipline goals are

· awareness of the range of knowledge in psychology

· competency in translating behavioral questions into the terms of scientific inquiry

· competency in reading and critically synthesizing the technical literature in psychology

· competency in quantifying and statistically analyzing behavior

· awareness of ethical issues in psychology.

2. The Capstone Course: Advanced Seminar.

A major change as a result of assessment was a revision of the existing capstone requirement, the Empirical Investigations (EI) courses. As noted in previous assessment summaries, the EI was an effective evaluation and capstone tool. However, even after reconfiguring it into a two-semester sequence, the EI had become too onerous a task with too many students failing to complete it in the time allotted. Research experience is vitally important for students who are capable and motivated to pursue doctoral study in psychology. However, many students do not meet those needs or goals. The result in those cases was a tremendous amount of “hand-holding,” leaving the capstone purpose in some doubt. Again, by consulting its goals and resources, the discipline developed a new capstone requirement. The Advanced Seminar course is intended to serve as a capstone experience for psychology majors, which means that its purpose is to both unify and provide a broader context for knowledge about the field of psychology gained throughout the undergraduate years. To accomplish this, students will:

· demonstrate their ability to read and critically synthesize primary source material;

· add to the collective knowledge and understanding of the seminar class members through preparation and active participation;

· investigate a topic within a broader topic (e.g., “addictive behaviors”) in depth;

· lead a seminar discussion focused on their chosen topic;

· communicate their findings in an extensive written report and public presentation.

For the seminar, course expectations and grading guidelines were developed as a discipline, but each section will be structured by individual faculty. Seminar students and all psychology faculty will be expected to attend the public presentations, allowing assessment across the discipline. Faculty will meet at the end of the academic year to evaluate the seminars. This type of capstone experience is also more in-line with other majors at UMM.

As noted, research experience is vitally important for students. One challenge will be for the psychology discipline to maintain the level of research opportunity with the required research experience removed from the major. Our intent is that quality and focus of research will improve with the removal of projects conducted just for the sake of getting them done. This will take extra effort on the part of faculty to encourage students to do research and success at adequately meeting students’ needs in this area will need to be assessed. One marker will be any change in the number of students engaged in UROP[7], REU, or other research experiences and another will be student participation in national and regional conferences.

3. Restructuring of major requirements

By monitoring advisee’s class choices since conversion to semesters certain deficiencies became apparent. Although psychology’s offerings covered the core areas of the field, and although the basic structure of the major remained consistent with what was once offered under a calendar based on quarters, the faculty found that students were taking a narrower selection of upper-level courses. Through comparisons with peer college’s psychology programs the faculty confirmed that the major

· required comparatively fewer course requirements

· offered markedly greater flexibility in what students may take rather than specifying credits to be taken within specific areas of the field.

The discipline’s response was to first increase the number of credits required for the upper-level electives from 16 to 20 credits. After monitoring that change for a year, it was evident that a more thorough evaluation and reorganization of the major was in order.

Over the course of several lengthy meetings , the discipline reviewed the 2002 report by the American Psychological Association’s Task Force on Undergraduate Psychology Major Competencies, comparison college’s catalogs, and its own goals and objectives. It found that the goals and objectives stated for its students were consistent with those set forward by the APA task force, but that students could complete the existing program and not meet all of them. Specifically, students could complete the undergraduate psychology major by only completing upper-level courses within a couple core areas of the field, thus specializing knowledge prematurely.

The discipline’s solution was to start from scratch, armed with the information gained. By matching objectives, faculty expertise, and credit requirements, it restructured the major, keeping the number of required credits at 42 while assuring that students will receive upper-level background across core areas of psychology. Students will now take at least one course from each core category: Learning and Cognition; Biological and Comparative; Personality and Clinical; Developmental; and Social and Applied psychology. Students still have some flexibility, choosing among 3 to 6 courses within any one category, and the ability to take additional courses from any category to meet the credit total.

4. Course embedded assessments

Over the past several years, one of the psychology faculty, in conjunction with director of UMM’s Academic Assistance office, has systematically evaluated student studying skills and performance in the introductory psychology course. By identifying differences between successful and less successful students’ approaches to readings and course materials, they have developed an electronic guide for studying for introductory survey courses and the findings are now used as part of course instruction in the introductory psychology course. Further, the research has helped inform and develop programs now in place at the Academic Assistance office.

General education categories spanned by the discipline

Psychology courses carry one of the following general education designators: SS, human behavior, social processes, and institutions; HDiv, human diversity; Sci-L or Sci, physical and biological sciences with or without lab; IP, international perspective; Hist, historical perspectives; E/CR, ethical and civic responsibility. Exceptions are directed study and research practicum, which carry no designator.

Sociology Discipline Assessment 2006-2007

Scope of assessment activities

___⇒__Course-embedded assessment

___⇒___ Pre- and post-testing

______ Outside the classroom

______ Across the discipline

Direct measures of student learning

___⇒__ Capstone experience

______ Portfolio assessment

___⇒__ Standardized tests

______ Performance on national licensure, certification or

preprofessional exams

______ Qualitative internal and external juried review of

of comprehensive senior projects

______ Externally reviewed exhibitions and performances in

the arts

______ External evaluation of performance during internships

Discussion and Description

Discipline goals, direct measures, and improved student learning

1. Sociology discipline goals

The sociology curriculum (along with support from anthropology courses) is designed to acquaint students with the concerns, theories, and methods of the science that focuses on groups, culture, and interpersonal relations of human beings. In addition to an introduction to sociology as a science, an effort is made to relate human values to the theories, methods, and data of sociology. Courses are designed to meet the needs of liberal arts students and those preparing for graduate school.

2. Sociology senior seminar[8]

Senior seminar is sociology’s capstone course and its principal assessment vehicle. In this course the student produces a thirty page research paper and gives a twenty-five minute presentation derived from it. In the past it has been a one semester course but beginning in the 2007-2008 academic year will span an entire year. The capstone objectives are:

i. to introduce the nature, uses, and objectives of research by turning an interest or idea into research questions and even problem solutions;

ii. to construct an argument by making claims and qualifying them appropriately;

iii. to think about and evaluate sources with a visionary and critical (yet constructive) mind;

iv. to discuss the complexities of planning, organizing, and writing a research paper;

v. to understand the ethical issues and problems in the research and writing process;

vi. to learn how to communicate research effectively and efficiently.

Two types of projects are possible:

i. projects involving data analysis;

ii. theoretical projects.

The former almost always involves human subjects, which requires the approval of the University of Minnesota Institutional Research Board. Obtaining permission from the IRB constitutes an external assessment of the fifth course objective, which all senior seminar students reach, not always on the first attempt.

One instructor is responsible for guiding students through the seminar and assessing their performance. Assessment is continuous as she meets with the class as a group and one-on-one with students throughout the semester.[9] She writes of her role in the third person as follows. “Prior to the presentation, Jennifer reads near-final drafts of all of the papers, meets with each student to critique and encourage their work and to offer suggestions for the presentation and paper.” She also reports, “All students showed growth in sociological understanding and critical thinking by the end of their senior seminar experience. Overall, I feel that this was a very successful year of senior seminar.”[10] Students are encouraged to present their results in the college’s annual undergraduate research symposium, and, in some instances, to submit their work to scholarly journals.

Although other social science faculty are invited to the presentations, their participation has been spotty. The instructor hopes to expand her colleagues’ contributions to the seminar, particularly in its assessment.

3. Course embedded assessment. Pre- and post-test.

Introductory sociology. This is the first course in the major and the course taken by many non-majors to satisfy the SS general education requirement (human behavior, social processes, and institutions). On the first and last day of class students were asked to give one sentence definitions of fifteen terms and were given the option of offering examples. Appendix B in the discipline report provides two years of data for classes of sixty-seven (falling to sixty-two by semester’s end) and forty-three students. In both instances, there is significant improvement from the first to last day, but in one instance it was observed that significant numbers of students were struggling with central concepts of the introductory course. The instructor took this observation into consideration the next time she taught the course.

Sociology of deviance. The instructor uses a pre- and post-test of eight questions on the first and last day of class.[11] Of particular interest are the changes in student responses to “How do you define ‘deviance’?” and “Who decides what is ‘normal’?” She tracks the responses of individual students and reports that over the past two years the class overall has shown improved understanding across the span of the semester.

Sociology of gender. The instructor for this course is the same as for the preceding, which is reflected in the similar format of the pre- and post-testing. For this course, there are twelve terms to define and one question to answer. As before, she tracks the responses of individual students and reports that over the past two years the class overall has shown improved understanding across the span of the semester.

Women in Muslim Society. The instructor lists three main objectives for the course and four methods for obtaining them. On the first and last days of class, students were asked to write on three questions of broad scope, one for each objective. The instructor reports that students moved from a state of almost complete ignorance to one where “their perspectives were broadening.”[12] She states that the course objectives were successfully reached. There is no data on individual objectives.

4. Course planning

The instructor of the capstone course hopes to discuss during the 2007-2008 academic year the introductory courses in sociology and anthropology with an eye to increasing their number and variety for both majors and non-majors.

General education categories spanned by the discipline

Sociology courses carry one of the following general education designators: SS, human behavior, social processes, and institutions; HDiv, human diversity; IP, international perspective; Envt, people and the environment; or E/CR, ethical and civic responsibility. Exceptions are directed study, qualitative research methodology, quantitative research methodology, tutorial in sociological theory, and independent project seminar I and II, which carry no general education designator.

---

[1] Quoted from the discipline assessment report, which is included in the appendices.

[2] Ibid.

[3] The full exercise is among the discipline papers in the appendices.

[4] Table 1 of the discipline report in the appendices gives the numerical results for each criterion.

[5] See the discipline report for the full lists.

[6] These tables and appendices are in the discipline report in the appendix to this report.

[7] Undergraduate Research Opportunities, an all-University program.

[8] Appendix A in the sociology discipline report in the appendices is the ten page syllabus for this capstone course.

[9] The schedule in Appendix A (see n1) details day-by-day where the student should be in the process, as well as the instructor’s involvement along with other college resources such as its research librarians and the English discipline’s writing room.

[10] See the main body of the discipline report in the appendices.

[11] See Appendix B of the discipline report.

[12] The report for this course is the last item in the discipline report.