Measuring Student Learning, Globally

WASHINGTON -- Nearly two years after the Bush administration said it would not participate in an international experiment aimed at developing a global assessment of student learning, the Organisation for Economic Cooperation and Development on Wednesday formally announced the launch of the effort -- with the full participation of the United States and the Obama administration.

The Assessment of Higher Education Learning Outcomes (AHELO) project aims to gauge whether it is possible to develop "reliable and useful comparisons of learning outcomes" that are valid across countries with different cultures and languages, said Richard Yelland, who heads the Education Management and Infrastructure Division at the OECD. The experiment will focus on producing three separate measures: one designed to measure general skills, and two in disciplines, economics and engineering.

The Australian Council for Educational Research will lead a consortium that will develop the discipline-specific tests. The general skills exam, meanwhile, will be developed by the Council for Aid to Education, to which OECD will pay $1.2 million to develop an international version of its Collegiate Learning Assessment, which has gained many institutional clients -- and its fair share of critics -- in the United States, where it has been framed as a tool for measuring the educational value that institutions add for their students.

OECD ultimately hopes to add a "value added strand" to its international assessments, but that will happen, Yelland told an audience at the Council for Higher Education Accreditation's international forum here Wednesday, only if the organization's research study concludes that it is possible to develop a single assessment that can cross the major divides presented by language, culture and country.

If you can prove that you can develop a tool to measure learning across cultures, "you should be able to run the test at two different times" to calculate "value added," Yelland said. "That's why we’re concentrating on [the toughest task of] proving cross-country validity.... We do not prejudge the outcome."

Details about the roughly $12.5 million project are still being developed, but Yelland said the experiment would aim to test about 200 students at roughly 10 institutions of diverse types in each of six countries: Finland, Italy, Japan, Kuwait, Mexico and the United States, with participation in the U.S. limited to four states: Connecticut, Massachusetts, Missouri and Pennsylvania. The students will be tested "near the end" of their either three- or four-year (depending on the country) bachelor's degree programs, and groups of experts will, by the early part of next year, decide whether the measures provide reliably comparable measures across the various countries.

If the "proof of concept" experiment is positive, and OECD decides to promulgate international learning outcomes measures that it believes are meaningful and reliable, "we will have a tool that will help us to help those who are responsible for higher education in the various countries," Yelland said. Given the vast sums of money that governments are investing in expanding the quantity of postsecondary education they provide -- roughly $1 trillion a year, about 1.5 percent of the global gross domestic product, Yelland estimated -- "it's probably worth spending a little bit of time and effort in supporting quality," he said.

How does testing support quality? "Diagnosis is the beginning of any improvement," Yelland said, and right now countries have no way other than international rankings -- which are based mostly on factors such as scholarly output and reputation, rather than educational factors -- to assess the quality of their institutions.

The U.S. Role

The United States' role in the project will go beyond having colleges in the four states participate; the governments in the AHELO initiative will also provide money for it, OECD officials said.

Education Department officials could not be reached for comment about the OECD announcement (it was a busy day, given President Obama's State of the Union speech last night), but an OECD news release included a statement from Under Secretary Martha J. Kanter that praised OECD for its leadership "to assess student performance on an international scale."

The U.S. participation represents a change of heart from 2008, when Bush administration officials more or less stunned higher education officials by saying they did not plan to join the OECD effort to develop an international higher education assessment. Education Department officials at the time said they "do not anticipate, as a U.S. government, funding ... a feasibility study" for such an assessment.

The decision was a shocker, as many in American higher education saw it, because the administration had pressed hard (too hard, in their eyes) to force colleges within the United States to use comparable measures of student learning. "There are certainly people who may have thought that the department is going to push for internationalization of the use of something like the CLA," Grover J. (Russ) Whitehurst, then director of the Education Department's Institute for Education Sciences and the U.S. representative on the OECD's education policy committee, said at the time.

Exactly what it means that the Obama administration is backing exactly such an approach, when the outcomes-obsessed Bush administration did not, probably depends on one's perspective. It could be seen as a sign that this administration is much more interested, on any range of matters, in international collaboration and partnership; it could also be evidence that this administration is just as interested in data-driven higher education accountability and testing as its predecessor -- if not more so.

Source: http://www.insidehighered.com/news/2010/01/28/oecd

Why Do I Need This Class?

A common challenge that educators face from students is the question “Why do I need this class?” Though we want to believe that students intuitively understand the importance of our subjects, often they do not. Many struggle to relate our disciplines to their lives. Without a sense of relevance, students experience a disconnection between our enthusiasm for our subjects and their experiences with it in and outside of class. This disconnection can hamper self-motivation, engagement and meaningful learning. Because of this dynamic, many of us now work to teach the relevance of our content disciplines. After attending an On Course workshop last year, I now employ a number of new tools in my history classes.

With them, I have been enhancing what I call a “subject-value pedagogy” that encourages students to make meaningful connections between discipline-based skills/knowledge and other aspects of their lives. In every way possible, I encourage my students to find personal value in the content of my courses. My students know this approach as “Why History?” skills. This article discusses how subject-value pedagogy can be used in any content area to increase students’ intrinsic motivation to learn it.

BEGINNING THE PERSONAL CONNECTION

Beyond usual active-learning methods, I now ask students to own and be the creators of their “Why History?” experiences. Starting in week one, I employ activities that reveal history as a practical tool for self-empowerment in the present. To be discipline specific, I adapted the wording of the On Course principles to read: “People from the past can role model for us how to take personal responsibility for our lives, have self-motivation, use self-management skills to get things done, be interdependent by working with others to achieve results, be self-aware about the ways we impact ourselves and others, look at life as a learning process, be realistic about managing our emotions, and believe in ourselves and abilities!” Then via written assignments, surveys, and discussions students reflect upon the manner in which they already value and employ “Why History?” skills. Though initially some do not see cross-pollination with other aspects of their lives, this approach sets a tone that shifts students’ perceptions of history from the impersonal and disconnected to the personal and connected.

SUSTAINING THE PERSONAL CONNECTION

Here is a quartet of “Why History?” activities that I employ:

1. Quotations: As a means of further prompting student thinking, a “Why History?” bulletin board displays quotations from and images of historical figures. For example, Philosopher Herbert Spencer reminds them that “The great aim of education is not knowledge but action”; while President Abraham Lincoln offers, “People are about as happy as they make up their mind to be.” At mid-term, my students then add their own group-created “advice to the future” along with their pictures. For example, one group metaphorically advised: “Today is homework for tomorrow’s exam.” Another group offered: “People who dwell too much on the past may come to realize that they are preventing their future.” During discussions, my students said this activity helped connect them to their role in shaping history.

2. Mission Statements: To encourage semester-long engagement, another classroom bulletin board, displays inspirational “Why History?” quotes. After reading and reflecting on words like Paul McElroy’s “We are the living link through which will be transmitted to the timeless future all that will be preserved of value from the ageless past,” students create personal “Why History?” mission statements. For example, one young man wrote: “I will use the information from the past, so I can apply what I learn to help better myself as well as other people who may need help in their own lives.” Another female student wrote: “I will strive to learn about the important events of history so I can relate past events with my choices, decisions, and ideas today.” Such statements are placed on the bulletin board, and at various times in the semester students share their “Why History?” thinking with the class. For fun, each person who shares places a star on his/her mission statement. From this activity, I have witnessed improved individual ownership of the class as well as an increased interdependence amongst students. This outcome has been anecdotally evidenced by their willingness to spontaneously offer “Why History?” thoughts and to ask others to share ideas. For example, one student shared that as a result of studying World War II, she was able to make a personal connection with a veteran at the nursing home where she worked. By asking questions and talking about what she knew, she felt she made a difference in the quality of his life.

To read the full article click here:
Source: --June Klees, Faculty, History, Bay College , MI
On Course Newsletter 2/2010

True Costs of Student Success

"Student success" programs of various types -- learning communities, first-year experience programs, and the like -- have proliferated on college campuses, driven by the reality that it's easier to keep current students than recruit new ones. The programs are popular, but as is true of just about all campus efforts these days, they are open to scrutiny about their effectiveness -- and their cost effectiveness.

Given that climate, many student affairs officials would probably be wary of looking too closely at what their programs cost and whether they provide a meaningful return on that "investment," for fear that the data, if they don't look good, might be used against them in the fight for resources. But putting those trepidations aside, 13 colleges -- as part of a project sponsored by Jobs for the Future and the Delta Project on Postsecondary Education Costs, Productivity and Accountability -- agreed to examine both the full costs of first-year retention efforts focused on first-generation and low-income students, and the extent to which their success in keeping students enrolled produces revenue to help pay for themselves.

A report on the Investing in Student Success effort, published this week by the two organizations, suggested that a majority of the programs had produced gains in retention that went a long way toward offsetting their costs. Most of the others could not complete the analysis, using a "cost return calculator" that includes a wide range of data, because they didn't have all the necessary cost and retention statistics (for students in the programs and for a comparison group).

But by looking at their first-year programs through the prisms of cost and return on investment, all 13 of the participating colleges experienced what the sponsors of the initiative call a "change in conversation" about those programs -- with some saying that they planned to apply similar scrutiny to other academic programs on their campuses. Officials at several of the campuses said they believed it was important to take financial considerations into account in assessing these programs, as long as they didn't overwhelm other, more qualitative factors, such as the programs' quality and the beneficial impact on students.

To read the full article click here:
Source: --Doug Lederman, Inside Higher Ed

Think Aloud Pair Problem Solving

As an instructor of English composition (mostly developmental), I have struggled to find ways to get students to engage themselves actively in their learning activities. Like many instructors, I get tired of seeing my students looking out the window, sending text messages, or doing some other equally non-productive activity during my class sessions. To diminish these behaviors, I have incorporated many active-learning strategies into my classroom, but I continue to see many non-engaged behaviors that suggest to me that students are still not actively engaging in the planned activities. I still often see students who refuse to speak or contribute in any way during small-group discussions, students who write one or two sentences and close their journals when I have asked them to write non-stop for five minutes, and students who are doing homework for some other subject during my class.

Because I know my students will be much more successful if they actively engage themselves in their learning experiences, I am constantly on the lookout for activities that will better facilitate active student engagement. In the activity I describe here, students work in pairs to solve a series of problems. Students have specific roles—problem solver and listener—that they alternate with each problem. The problem solver “thinks aloud,” verbalizing the steps he or she takes to solve the problem. The listener listens carefully, following the steps taken by the problem solver, attempting to understand the reasoning behind the steps, and offering suggestions if necessary.

Although I used the activity with grammar rules, it could easily be adapted to suit the needs of any instructor in any discipline. Approximate time needed: 30-45 minutes.

PURPOSE

• To have students actively engage in the learning process
• To have students learn to identify relevant information and apply it in the solution of a problem
• To have students learn and practice problem-solving strategies

SUPPLIES/SET UP

• A set of problems to solve (I used problems from the grammar workbook I use in my class
• Plenty of writing instruments for writing on the board (dry erase markers for whiteboards, chalk for blackboards, etc.)

To read the full article click here:
Source: --Adrienne Peek, Faculty, English, Modesto Junior College , CA
On Course Newsletter 2/2010

Why You Shouldn’t Use PowerPoints in (Most) Online Courses

Sigh. Where to begin? There are so many reasons why using PowerPoint for online courses is a Bad Idea. PowerPoint is just a tool, of course, but it’s so often the wrong tool for the job, especially in teaching online. A hammer is only a tool, but in the wrong hands, well, it makes a mess of things. So it is with PowerPoint.

It’s not that PowerPoint—henceforth referred to as PPT—can’t be used effectively for teaching online. It’s just that most people have developed deplorable PPT habits and now believe that textually dense PPT slides, cheesy animated transitions, and gaudy 3-D graphs are de rigueur for teaching in the classroom and therefore are the perfect choice for online courses, too. To that I say “Nuh-uh. Not. FAIL!” Who else says so? Well, these guys:

To read the full article (with videos) click here:
Source: http://nauelearning.wordpress.com/2009/10/21/nopptonline/

Witch's Broom

I teach general chemistry at a community college. Most science and engineering majors take the course in their first semester and need to earn a high grade to gain admittance to a four-year school in their major. Nonetheless, the success rate in this course remains very low, with an attrition rate between 30-50%.

One of the reasons that many of these students are unsuccessful is their beliefs about learning. Many believe that learning is a passive process, that learning is a solitary activity, and that if they are smart they won’t need help. They also greatly underestimate the time they need to devote to studying in order to be successful. Experience is a slow teacher, and by the time they realize these beliefs are inaccurate, most have gotten too far behind and drop out. In the past I have attempted to address these beliefs head on. I’ve spent a lot of the first class lecturing students on what it takes to be successful in chemistry. I even designed a handout that clearly explains how to be successful in the course. As you might guess, my lecture and handout were largely ignored.

Last semester, I decided to try a first-day activity that I hoped would catch students’ attention, foster interdependence, and make them aware of the active nature of learning.

This activity can be used in any course. It can be done in as few as 20 minutes, but can be extended depending on the depth of discussion you choose to facilitate.

PURPOSE

• To create an attention-getting first-day experience
• To help students become aware of the active nature of learning
• To foster interdependence between and among students

SUPPLIES

• Four-foot length of string or yarn, ends tied to create a loop, one per student
• Instructions for Witch’s Broom, one per student (appended below in Support Documents)
• Two questions for student reflection written on board (see directions below)

To read the full article click here:
Source: --Kirsten Casey, Faculty, Chemistry; Anne Arundel Community College, MD
On Course Newsletter 1/2010

10 Warning Signs of a Bad Professor

Many students are heading back for the second semester of college this week. How the semester goes will depend heavily on the quality of the courses they've chosen. Many students will consult sites such as www.ratemyprofessors.com, their college's own evaluation systems (when public), and the general scuttlebutt from their real and virtual friends. But it's always better to size the professor up yourself by attending the first couple of lectures, then dropping the course if you think the professor is bad. But how do you tell? Here are our 10 surefire signs that your prof's a dud—and that you should get out while there's still time:

1. The professor is boring. Even in the very first classes, you can tell if the professor presents the material in an interesting way. Be especially alert for professors who stand up there and read large sections of the lecture from their notes. If it's a snoozer in Week 1, it's going to get excruciating by Week 15.

2. The professor is bummed out. If a professor comes in on the first day of classes already grumbling about how much he or she hates teaching this course, how much he or she would like to be teaching at a better college, or how teaching is a waste of his or her time (because research is where it's at), don't expect things to get any better as the course progresses. Rule of thumb: Bummed out to start gets more and more bummed out, exponentially, as the weeks drag on.

3. The professor doesn't give out a syllabus—or hands out a one-paragraph syllabus that is just the course description from the Web. Professors who don't distribute a detailed syllabus probably don't actually know what they're going to do in the class this semester.

4. The professor isn't clear about the requirements and how much they count. Professors who don't have a clear and comprehensible idea about how the grading will be handled can end up springing all sorts of wacky or inconsistent grading plans on the students. Often students in this sort of class never know how they're doing during the semester and end up with unpleasant surprises at the end.

5. The professor assigns an undoable amount of work—or no work at all. In the first case, the professor has unreasonable expectations of the students—the kiss of death for any student trying to do well in the course. And while the alternative—little or no work—might seem tempting, it's likely to result in a course where you learn nothing.

6. The professor has incredibly petty rules. Bail out if you encounter a syllabus with page upon page of rules dealing with everything from the use of cellphones to whether you can wear caps to an exam; how to address the professor; when you can enter the room and when you can exit the room; policies about eating, drinking, and using the bathroom; 25 acceptable reasons for an extension and 53 unacceptable reasons; grade penalties for lateness timed to the half-hour, etc. Look, you're taking a course, not rewriting the healthcare system.

7. The professor can't fill the whole class period. Lots of professors hold a short session the first day: They introduce themselves, go over the syllabus, and call it a day. But if class is let out early the whole first week, you can be pretty sure that the professor is inexperienced, is a bad planner, or, worst of all, doesn't really give a damn about the course. Sure, you'd like to blow it off early every day. But why cheat yourself out of the education you've paid for?

8. The professor seems unsure about the material. Professors who present their lectures in a halting or tentative way could well be professors who aren't on top of the course content. You might think that colleges would hire only people who really know the material backward and forward, but you'd be wrong. It's not at all uncommon for faculty to be saddled with a course in which they have no expertise. Why should you be saddled with it, too, if you've got a choice?

Extra sign. If the professor says he or she is going to "learn the material with you," run for the exits. That's professor-speak for "I don't know my a** from my elbow about this stuff."

9. The professor presents the material in a confused way. Every lecture at college should have a clear structure: an introduction, a main part, and a conclusion. If your professor's lectures are all over the place and you can't figure out what the main points are—or when and why the professor has moved from one point to the next—something is definitely wrong. Example: The professor isn't able to explain the stuff in a way the students can understand. In street language, he or she can't teach.

10. The professor never involves the students. If a professor attends only to his or her notes and never even looks at the students—or never pauses to invite or accept questions—it's not a good thing. A good class is a dynamic class, and a good professor engages with the students.

BY: LYNN F. JACOBS & JEREMY S. HYMAN • JANUARY 6, 2010 • US NEWS &WORLD REPORT

Source: http://www.usnews.com/blogs/professors-guide/2010/01/06/10-warning-signs-of-a-bad-professor.html