Economics Discipline Assessment 2006-2007

 

Scope of assessment activities

         ___⇒__Course-embedded assessment

                     ___⇒___ Pre- and post-testing

         ______ Outside the classroom

         ______ Across the discipline

Direct measures of student learning

         ______ Capstone experience

         ______ Portfolio assessment

         ______ Standardized tests

         ______ Performance on national licensure, certification or

                     preprofessional exams

         ______ Qualitative internal and external juried review of

                     of comprehensive senior projects

         ______ Externally reviewed exhibitions and performances in

                     the arts

         ______ External evaluation of performance during internships

              

Discussion and Description

Discipline goals, direct measures, and improved student learning

 

         1. Economics discipline goals. The economics curriculum is designed to ensure that students

á      understand the nature and functioning of the market system.

á      are able to define criteria for assessing efficiency in the provision of goods and services.

á      investigate and assess the operation of economic institutions.

á      are able to evaluate alternative policies intended to enhance economic outcomes.

á      develop competence in quantitative methods and computing methods.

á      are able to conceptualize and analyze problems using the tools of economic theory, and communicate the results.

á      are competent in oral and written communication.

á      are adequately prepared for graduate or professional school.

 

         2. Discipline goals and course work

         The assessment plan relates discipline goals to the economics courses where they will be met.

 

         3. Course-embedded assessment

         Money, Banking, and Financial Markets. In this upper division class, the instructor identified two broad learning objectives that students should meet, and for each objective established three criteria for determining whether students had met them. The assessment tool was a multi-part in-class essay question admininstered two months into the semester and again near the end of the semester. The instructor then ranked each studentÕs performance on each criterion as good, fair or poor. The results[1] show improvement in student learning that is more often dramatic than not.

 

General education categories spanned by the discipline

 

            Almost all economics courses bear one of the following general education designators: SS, human behavior, social processes, and institutions; IP, international perspective; HDiv, human diversity; M/SR, mathematical/ symbolic reasoning; Hist, historical perspectives; or Envt, people and the environment. Directed study and the seminar for social science majors bear no general education designator.

Education Discipline Assessment 2006-2007

 

Scope of assessment activities

         ___⇒__Course-embedded assessment

                     ___⇒___ Pre- and post-testing

         ___⇒__ Outside the classroom

         ___⇒__ Across the discipline

Direct measures of student learning

         ___⇒__ Capstone experience

         ___⇒__ Portfolio assessment

         ___⇒__ Standardized tests

         ___⇒__ Performance on national licensure, certification or

                     preprofessional exams

         ______ Qualitative internal and external juried review of

                     of comprehensive senior projects

         ______ Externally reviewed exhibitions and performances in

                     the arts

         ______ External evaluation of performance during internships

              

Discussion and Description

Discipline goals, direct measures, and improved student learning

 

         1. Discipline overview

         The curriculum is divided into three subcurricula, Education, Elementary Education, Secondary Education, and Education. Elementary Education is offered as a major, but Secondary Education is a licensure program in which the student majors in a discipline from one of the other[2] Divisions: Humanities; Science and Mathematics; or Social Sciences.

 

         2. Discipline goals

         2a. Elementary and Secondary Education

         The goals for Elementary and Secondary Education are designed to help students (future teachers) to

á      acquire the knowledge, skills, and dispositions necessary to being a competent teacher

á      understand central concepts, tools of inquiry, and structures of disciplines taught in schools

á      understand children and adolescents and their individual and group behavior

á      plan and implement instruction adapted to learners of diverse backgrounds and abilities

á      communicate effectively

á      encourage critical thinking and problem solving

á      use formal and informal methods of assessment

á      collaborate with parents/guardians, families, school colleagues, and the community in an ethical manner.

These goals are based on the ten Standards of Effective Practice of the state of Minnesota.

         2b. Education

         The courses are designed to offer students the opportunity to study education and its role in society.

 

         3. A three-fold assessment strategy in Elementary and Secondary Education

         The discipline uses both internal and external measures of the degree to which students achieve the goals listed above. Three principal assessment tools are

á      PRAXIS II exams

á      summative evaluation scores for the final student teaching experience

á      key assignments in the capstone course: portfolio assessment

 

         4. PRAXIS II exams

         There are two parts to these standardized, external exams, the PLT (Pedagogy, Learning and Teaching—or the pedagogy part for short) and the content part. The state requires that students pass both parts for licensure. In 2007 UMM had a 100 % pass rate for both parts (29 students in Elementary Education and 25 in Secondary).[3] During the 2002-2007 period, UMM students took content exams in eighteen areas, with pass rates exceeding state pass rates in fifteen of them, although in some instances the differences in rates are small and some small sample sizes vitiate the comparison.[4]

         Performance on PRAXIS II may also signal areas where student learning is problematic. One example is discussed in the disciplineÕs report, which details the problem, describes the facultyÕs response, and documents the improvement in student learning. Another issue being addressed by the Elementary Education faculty is the presence of two students in the cohort of 2009 for whom English is a second language, the issue being whether PRAXIS II will fairly assess their learning.


 

         5. Summative evaluation of student teaching[5]

         Elementary and secondary education students complete eleven weeks of student teaching as their final field experience. During that field experience, they apply the knowledge and skills studied in the programs. In this experience, all ten Standards of Effective Practice are implemented and assessed.  Cooperating teachers and university supervisors complete at least three formative evaluations that highlight strengths in performance and assist the student in identifying weaknesses. Goals are established, additional coaching is provided, and students are given the opportunity to address areas of concern. The summative evaluation assesses all of the program goals and is completed by cooperating teachers and university supervisors. Data from final field experience evaluations for the 2006-2007 cohort reveal that all but one student met minimum proficiency for licensure (please see Table One). No other student received less than an ÒaverageÓ rating on any part of their field experience evaluation. In fact, most earned high marks from cooperating teachers or university supervisors.

 

         6. Key assignments from the capstone course: portfolio assessment

         The goals of the capstone course are to

á      facilitate professional reflection

á      enable students to explore professional issues related to teaching

á      assist students in evaluating the effects of their professional choices and actions on students, parents, other professionals, and the larger learning community.

         The primary assessment of student learning in the capstone course is the professional portfolio created by students. Students begin creating this portfolio when they enter the program and continually revise it throughout. The portfolio has ten sections, one section for each of MinnesotaÕs Standards of Effective Practice—standards in which students must demonstrate competency before being licensed as teachers. For each standard, students write an essay that describes their growth and development in the standard, provides evidence of that growth and sets new goals for deeper understanding. This process again exemplifies the formative assessment built into the teacher education program. Students have multiple opportunities to write, reflect on feedback, and reconstruct their portfolio throughout the program. In this course, the students write their final statements. Each faculty member is responsible for evaluating 10-12 professional portfolios. Prior to the evaluation period, faculty members engage in a reliability session to ensure fair and reliable grading practices across faculty members. In this session, faculty read and grade sample essays and discuss the reasons for their grades and discuss any discrepancies. The process continues with multiple readings and discussions until the faculty are grading in a manner consistent with one another.


         7. Course-embedded assessment

         All courses in the elementary and secondary education programs are crafted around the ten Standards of Effective Practice. Lectures, readings, and assignments are linked to specific standards and the links are usually recorded on the syllabus. The courses also are based on mastery learning. This means that students must perform all tasks at a proficient level. If a student does not successfully master a task, he or she continues to work on this task and repeat an assignment until it is mastered.

         An example of assessment in one course is in Appendix 1 of the Education disciplineÕs report. Assessment is built around a teaching and learning strategies mini-unit assignment. The final goal is to have the student plan and teach the mini-unit to his or her practicum students, and then to assess their learning. The studentÕs mastery in turn is evaluated by the instructor by means of a scoring rubric that the student has seen in advance. The route to attaining this goal is laid out in an eight-step plan.

 

         8. Students who fail to meet the requirements

         The Education disciplineÕs report notes that its Òdata clearly show that most of the students far surpass the minimum requirements set by our own program and those of the state and national accreditation agencies.Ó The disciplineÕs faculty continues to seek ways to assist the rare student who does not self-select out of the program but is struggling to meet its requirements.

 

General education categories spanned by the discipline

 

            Many Education courses but by no means all carry one of the general education designators: IP, international perspective; Hum, communication, language, literature, and philosophy; E/CR, ethical and civic responsibility; or FL, foreign languages. Only two courses in Elementary Education and only two in Secondary carry designators, one IP, and one HDiv, human diversity, in each of the subcurricula.

English Discipline Assessment 2006-2007

 

Scope of assessment activities

         ___⇒__Course-embedded assessment

                     ___⇒___ Pre- and post-testing

         ______ Outside the classroom

         ___⇒__ Across the discipline

Direct measures of student learning

         ___⇒__ Capstone experience

         ___⇒__ Portfolio assessment

         ______ Standardized tests

         ______ Performance on national licensure, certification or

                     preprofessional exams

         ______ Qualitative internal and external juried review of

                     of comprehensive senior projects

         ______ Externally reviewed exhibitions and performances in

                     the arts

         ______ External evaluation of performance during internships

              

Discussion and Description

Discipline goals, direct measures, and improved student learning

 

         1. English discipline objectives. Students learn to

á      discuss, orally and in writing, what they have read: how the author has structured the text, and how literary language achieves its effects and directs the readersÕ response to the text.

á      be effective critical and imaginative readers and writers.

The English program engages students in the study of primarily British and American literature of different periods, with an emphasis on various approaches to literary study.

 

         2. Assessment across the discipline[6]. This is achieved through a three-fold approach of assessing

i.      the college writing program

ii.     the gateway course to the major

iii.   the capstone course.

In all three instances, there are variations on the pre-test/post-test method (testing = annotating or writing) and strategies for improving student learning.

 

         3. Assessment of College Writing.

         3a. Overview.

         College Writing fulfills a general education requirement for all UMM students. The current course goals are the result of English faculty deliberations in 2002-2003 that align their concerns with those expressed in a cross-disciplinary survey of the college faculty.

         3b. Three broad goals. Students should be able to

á      state an argumentative thesis clearly at the beginning of a paper.

á      analyze (rather than merely summarize) evidence for that thesis.

á      appropriately revise the paperÕs content and/or form in response to peer and instructor feedback.

         3c. Specific learning objectives. By the end of the course students should be able to

á      understand and recognize the basic conventions of effective academic writing.

á      articulate a specific and argumentative thesis.

á      develop and organize an argument.

á      supply and analyze appropriate evidence in support of a claim.

á      understand citation norms and use an appropriate citation format.

á      paraphrase, summarize, and effectively quote sources.

á      locate sentence-level errors in their writing and find answers/help in a reference book.

á      understand writing as a process (planning, drafting, revising, editing).

á      make meaningful and substantive revisions in their own work.

á      offer constructive comments, both in writing and orally, on peersÕ work.

         3d. Portfolio assessment.[7]

         Students wrote three drafts of each required paper. The final portfolio consisted of the third draft of all papers. For each draft and for each of the three broad goals in 3b, the instructor rated each student in one of three categories: having mastery; having competence; or lacking competence. This method measured the degree of achievement of course goals and the degree of improvement for each student. The set of ratings for the semester measured achievement and improvement for the class as a whole.

         3e. A conclusion and moving forward.

         The instructor concludes that Òour current methods of assessment in these areas are generally successful.Ó She states that two English faculty have been awarded a grant to revise the course syllabus to Òresult in more effective cross-disciplinary preparation for our students.Ó

 

         4. Assessment of the gateway course. Introduction to Literature.

         4a. Overview.

         ÒThis course was introduced in 2003 to address a substantial shortcoming in studentsÕ preparation for upper-level English courses.Ó[8] The vehicle for addressing this shortcoming is poetry analysis, a task that students find particularly difficult.

         4b. Course learning objectives.[9] They are to improve studentsÕ

i.      ability to READ carefully and actively.

ii.     ability to understand and discuss literature analytically, using the conventional terminology.

iii.   understanding of and ability to implement processes of making a critical argument.

         4c. Assessment tool.

         Students annotate a poem at three times during the semester. Each time and for each learning objective the student is rated as having mastery, having competence, or lacking competence. The instructor discusses the first annotation with the students, who then practice annotating in class with instructor feedback. The second and third annotations measure improvement and the degree to which the learning objectives have been met.[10]

         4d. Future issues.

         The English discipline is discussing, as any large discipline should, the problems that arise from having multiple instructors in the course who approach the material in slightly different ways. Another is issue is to make certain that students, who by and large are reaching the learning objectives, retain those abilities. The discipline is discussing whether an annotation exercise should be employed early in all of the survey courses to reinforce those skills.

 

         5. Assessment of the capstone course. Research seminar.

         5a. Overview.

         All English majors must complete at least one research seminar from the list of ten in the 2007-2009 University of Minnesota, Morris Catalog.

         5b. Learning objectives. The seminar adds two to those listed for the discipline:

á      the development of sophisticated research skills.

á      the ability to engage publicly with current debates in the field.

         5c. Assessment tools.

         In reaching the first objective, students must produce an annotated bibliography, which is judged on the number and quality of the sources, and on the quality of the annotations.[11] The bibliography provides the basis for writing a substantial research essay, about ten pages in length.

         To reach the second, students must give a fifteen to twenty minute oral presentation in the public English Research Symposium, which follows models of conferences attended by English academics. This includes being a member of a panel.

         5d. Improving student learning.

         To address weaknesses in preparing annotated bibliographies, finding appropriate sources will be introduced into junior level survey courses, as well as the art of writing annotations, and more time will be devoted to sources in the seminar itself. Problematic bibliographies received faculty and conference attention. Faculty critiqued drafts of essays.

        

General education categories spanned by the discipline

 

            Three English courses carry the CW, college writing, general education designator. Almost all others carry one of the following: Hum, communication, language, literature, and philosophy; HDiv, human diversity; ArtP, artistic performance; or Envt, people and the environment. The only exception is directed study, which carries none.

French Discipline Assessment 2006-2007

 

Scope of assessment activities

         ___⇒__Course-embedded assessment

                     ___⇒___ Pre- and post-testing

         ___⇒__ Outside the classroom

         ___⇒__ Across the discipline

Direct measures of student learning

         ___⇒__ Capstone experience

         ______ Portfolio assessment

         ___⇒__ Standardized tests

         ___⇒__ Performance on national licensure, certification or

                     preprofessional exams

         ______ Qualitative internal and external juried review of

                     of comprehensive senior projects

         ______ Externally reviewed exhibitions and performances in

                     the arts

         ______ External evaluation of performance during internships

              

Discussion and Description

Discipline goals, direct measures, and improved student learning

 

         1. French discipline learning objectives

á      An introduction to the four skills of communication in French: speaking, listening, reading, and writing.

á      Intermediate French: raising the level of sophistication in communication and broadening the cultural base to include French and francophone cultures.

á      Continuation of objectives 1 and 2; mastery of literary texts and initiation into critical approaches and sharpening analytical skills.

á      Application of objectives 1 through 3 above to original work in the field; post-baccalaureate employment and graduate study.

 

         2. Two phases of assessment

         The Iowa Placement Test is used to assess the first two years of the major, i.e., the beginning and intermediate phases. The last two years, the advanced phase, are assessed using the American Council for the Teaching of Foreign Languages (ACTFL) Proficiency Guidelines for Speaking and Writing.

 

 

         3. Senior seminar

         The senior seminar is the disciplineÕs capstone experience, in which the student produces a substantial scholarly or creative work at a level appropriate for an undergraduate. It requires intensive revision of a major writing assignment and a public presentation given in French.

 

         4. First phase

         The Iowa Placement Test (IPT) is used in a pre-test/post-test format. It is administered to all students who have had previous experience with French and wish to enroll in French. It is administered again at the end of second semester of beginning French, and for a third time at the end of the second year, i.e., at the end of the second semester of intermediate French. The IPT assesses reading, grammar, and listening comprehension skills.

 

         5. Second phase

         5.1 Writing proficiency. The third year of the major requires one semester courses in conversation and composition, and in reading and analysis of texts. Towards the end of the second semester of the third year, a writing sample is obtained and assessed using ACTFL guidelines. Students complete a second writing sample in the capstone course that is assessed with respect to the same guidelines and is compared to the third year sample.

         5.2 Speaking proficiency. The oral presentation in the senior seminar is assessed using ACTFL guidelines.

 

         6. Outcomes

         6.1 First phase.

         Student IPT scores are compared to a standard score representing the studentÕs preparedness for the next course in the major.[12]

         6.2 Second phase

         On the basis of assessment, speakers and writers are placed in one of ten categories: superior; advanced (high, mid or low); intermediate (high, mid or low); and novice (high, mid or low).[13]

 

         7. Improving student learning

         Assessment in the French discipline has led to the following initiatives.

á      Four new courses will weave grammar instruction through all four years of coursework. These are courses in phonetics, translation, advanced grammar, and business French.

á      A system of one credit ÒmaintenanceÓ courses will allow students to practice oral skills in a more intimate setting.

á      A professionalization component has been introduced into the senior seminar for which students prepare resumes, curriculum vitae, and cover letters in French, participate in mock interviews in English, and research (and in some instances even apply for) jobs using their French skills.

These are recent initiatives whose efficacy will be assessed.

 

         8. Possible forms for future assessment

         At the present time, only language proficiency is assessed systematically. However, nearly all of the advanced courses in the curriculum have a cultural focus, which is also a signficant component of the first and second year courses. The discipline will be discussing whether and how to assess the acquisition of cultural knowledge.

 

General education categories spanned by the discipline

 

            Almost all French courses bear one of the following general education designators: FL, foreign language; IP, international perspective; Hum, communication, language, literature, and philosophy; or Hist, historical perspectives. Directed study and senior seminar carry no general education designator.

 

 


[1] The numbers are in the economics discipline report in the appendices.

[2] Education is the fourth academic division at UMM.

[3] See Table One of the Education disciplineÕs report in the appendices. This table also summarizes results for all three assessment tools for the class of 2007.

[4] Table II in the discipline report.

[5] The paragraphs in ¤ 5-7 have been pasted almost entirely from the Education discipline report in the appendices.

[6] The complete assessment report is in the appendices.

[7] The discipline report in the appendices gives numerical details of assessment in one eighteen-student section of college writing.

[8] Ibid.

[9] These are abridged from the full text in the appendices.

[10] The appendices contain numerical results for sections from 2005 and 2007.

[11] The discipline report discusses qualitatively the strengths and weaknesses of these bibliographies.

[12] Results for the 2006-2007 academic year are in the French discipline report that is in the appendices.

[13] See appendices for detailed results. For writing proficiency, senior seminar students ranged from advanced-high to intermediate-high. For oral proficiency, the same group ranged from advanced-high to intermediate-mid. The author of the report in the appendices notes that Òstudy abroad is not a reliable predictor of higher levels of proficiency in writing,Ó but cautions that the sample size is small. The predictive power of study abroad is better for oral proficiency, but not clear-cut.