The good old “I’m a filter, not a pump” keeping the not so good students down where they belong approach.  “I’ll just cater for the good students”.

A rather cynical comment from Bernard Pliers, actually on Maths education:

Evaluating the trial in the Weiman study had three dimensions

• Student engagement
• Post-test
• Attendance.

ENGAGEMENT

This fascinated me. So I reproduce in full from the supporting notes:

The engagement measurement is as follows. Sitting in pairs in the front and back sections of the lecture theatre, the trained observers would randomly select groups of 10-15 students that could be suitably observed. At five minute intervals, the observers would classify each student’s behavior according to a list of engaged or disengaged behaviors (e.g. gesturing related to material, nodding in response to comment by instructor, text messaging, surfing web, reading unrelated book). If a student’s behavior did not match one of the criteria, they were not counted, but this was a small fraction of the time. Measurements were not taken when students were voting on clicker questions because for some students this engagement could be too superficial to be meaningful as they were simply voting to get credit for responding to the question. Measurements were taken while students worked on the clicker questions when voting wasn’t underway. This protocol has been shown by E. Lane and co-workers to have a high degree of inter-rater reliability after the brief training session of the observers

E. Lane is referred to, but not referenced but is sure to be Erin Lane.

There is a diagram from one of her studies which is looking an Earth and Ocean Science class.  Physics is not the only discipline seeking approaches to improve engagement:

From: www.cwsei.ubc.ca/SEI_research/files/Geo_Ocean/Lane_QuantifyingStudentBehavioralEngagement_poster.pdf

From the report:

In the experimental section, student engagement nearly doubled

THE TEST

The test questions for this topic were agreed after the week of teaching, both instructors agreeing it was a good test of the objectives. (Whew!!) From the paper:

The average scores were 41 (+/- 1%) in the control section and 74 (+/- 1%) in the experimental section. Random guessing would produce a score of 23%, so the students in the experimental section did more than twice as well on this test as those in the control section

ASIDE: all the questions are included in the online report. They are HARD questions.

ATTENDANCE

During the week of the experiment, engagement and attendance remained unchanged in the control section. In the experimental section, student engagement nearly doubled and attendance increased by 20% (Table 1). The reason for the attendance increase is not known

What is significant

It seems obvious: pay more attention and come to class more and you learn better.  Maybe.  There is a complex relationship between interest, motivation, effort, time on task, the right kind of task (etc)

In summary: two teachers taught a well defined subject to two groups to all intents and purposes the same.  Different approaches.  They both tried hard.  One group’s results were far superior to the other.

What does this mean? you may ask.

The lecturers: Instructor (A), a successful lecturer (who had won a teacher of the year award, and had good student evaluations) and a post-doc tutored by Weiman.
Both teachers gave it their best shot

Instructor A and L.D. had agreed to make this a learning competition

ASIDE: this in itself is an astounding opportunity: an instructor (A) with a history of good evaluations etc etc agreeing to this. I reckon his/her name and identity will emerge in due course hopefully NOT on a sleazy talk show, and we’ll learn some more about this project.

The course: traditionally, a physics course is divided into topics, with some building on each other.  The first part of the course was taught traditionally.  In the study the final topic (and the subject of the study) is a complete unit, Electro-magnetism.

L.D. and instructor A agreed beforehand what topics and learning objectives would be covered

The teaching: no formal lecturing at all in the experimental section. Instead:

The instructional approach used in the experimental section included elements promoted by CWSEI and its partner initiative at the University of Colorado: preclass reading assignments, preclass reading quizzes, in-class clicker questions with student-student discussion (CQ), small-group active learning tasks (GT), and targeted in-class instructor feedback (IF). Before each of the three 50-min classes, students were assigned a three- or four-page reading, and they completed a short true false online quiz on the reading

No new gadgets were used. Clickers has been used the whole course.
Pre-reading: the only change in the control section was the requirement to read the text in advance.

The populations (267, 271): using various statistical measures these were essentially identical eg same mean (+/- 1%) in mid term exam.

What is significant here

The value of pre-reading.  Other studies show this has an effect on learning.  We could say “this is obvious” but traditionally it has been hard to convince students of the value of actually doing it – and now we have ways to encourage this to be more of a regular habit.  The online testing with self marking helps.

This list from the ROTP project, used in teaching/teacher evaluation:
“The Reformed Teaching Observation Protocol (RTOP) was developed as an observation instrument to provide a standardized means for detecting the degree to which K-20 classroom instruction in mathematics or science is reformed per the national science and mathematics standards.”

1. The instructional strategies and activities respected students’ prior knowledge and the preconceptions inherent therein.
2. The lesson was designed to engage students as members of a learning community.
3. In this lesson, student exploration receded formal resentation.
4. This lesson encouraged students to seek and value alternative modes of nvestigation or of problem solving.
5. The focus and direction of the lesson was often determined by ideas originating with students.
6. The lesson involved fundamental concepts of the subject.
7. The lesson promoted strongly coherent conceptual understanding.
8. The teacher had a solid grasp of the subject matter content inherent in the lesson.
9. Elements of abstraction (i.e., symbolic representations, theory building) were encouraged when it was important to do so.
10. Connections with other content disciplines and/or real world phenomena were explored and valued.
11. Students used a variety of means (models, drawings, graphs, concrete materials, manipulatives, etc.) to represent phenomena.
12. Students made predictions, estimations and/or hypotheses and devised means for testing them.
13. Students were actively engaged in thought-provoking activity that often involved the critical assessment of procedures.
14. Students were reflective about their learning.
15. Intellectual rigor, constructive criticism, and the challenging of ideas were valued.
16. Students were involved in the communication of their ideas to others using a variety of means and media.
17. The teacher’s questions triggered divergent modes of thinking.
18. There was a high proportion of student talk and a significant amount of it occurred between and among students.
19. Student questions and comments often determined the focus and direction of classroom discourse.
20. There was a climate of respect for what others had to say.
21. Active participation of students was encouraged and valued.
22. Students were encouraged to generate conjectures, alternative solution strategies, and ways of interpreting evidence.
23. In general the teacher was patient with students.
24. The teacher acted as a resource person, working to support and enhance student investigations.
25. The metaphor “teacher as listener” was very characteristic of this classroom.

From Anton Lawson’s site.

The Reformed Teaching Observation Protocol (RTOP) was created by the Evaluation Facilitation Group (EFG) of the Arizona Collaborative for Excellence in the Preparation of Teachers (ACEPT). It is an observational instrument designed to measure “reformed” teaching.

The initial development of the RTOP is now complete, and the instrument is being widely circulated.  Consequently, there is a need for a manual that contains the more technical information about the RTOP that might be used by scholars and researchers. This document is designed to fill that need. The theoretical constructs that guided the design of the instrument are presented here, as are reliability and validity information. In addition, the results of an exploratory factor analysis of the RTOP are presented.

I am reading this in the light of John Hattie’s work, and the wondering: what is evidence based research when it comes to teaching and learning?

Today, a link crossed my monitor that mentioned

Russ Edgerton’s white paper on Pedagogies of Engagement?  The commonly referred to link appears to be inactive ( www.pewundergradforum.org/wp1.html)

I checked out the phrase and discovered several interesting pages.

1. http://www.ce.umn.edu/~smith/docs/Smith-Pedagogies_of_Engagement.pdf

Prior to Edgerton’s paper, the widely distributed and influential publication called The Seven Principles for Good Practice in Undergraduate Education [2] stressed pedagogies of engagement in concept. Three of the principles speak directly to pedagogies of engagement, namely, that good practice encourages student-faculty contact, cooperation among students, and active learning.

2. http://www.carnegiefoundation.org/publications/sub.asp?key=452&subkey=612&printable=true

One of Russ’s arguments focused on something he called “pedagogies of engagement” — approaches that have within them the capacity to engage students actively with learning in new ways. He wasn’t talking only about service-learning, though service learning was an example; he was talking about an array of approaches, from problem-based and project-based learning to varieties of collaborative work and field-based instruction. Russ used the rubric “pedagogies of engagement” to describe them all.

3. http://www.bgsu.edu/cconline/Civil_War/CWlearnercentered.htm

“Engagement,” framed within the theoretical concerns of social and cognitive development, seems to be largely about a student’s “maturity.” So, a student is engaged when s/he shows or self-reports gains in:

• “personal development, academic achievement, civic responsibility, [and] career exploration” (Billig and Eyler)
• personal development such as sense of personal efficacy, personal identity, spiritual growth and moral development (Vanderbilt review 2000)
• interpersonal development and the ability to work well with others, leadership and communication skills (Vanderbilt review 2000)

To put it bluntly, where’s the fun in that? (Emphasis and italics added by me)

Just a taster really. How do you know someone is engaged? Does it REALLY affect learning?

I should have been posting on this last week as I read Bains superb book “What the Best College Teachers Do“. Here is something adapted from what I wrote for last week’s UCTL news sheet (FLAB):

‘What the Best College Teachers Do’ (Ken Bain)

Mike has found this quite a remarkable book, and loaned me a copy.

• http://www.montclair.edu/center/Bain.html

The book is a report on a fifteen-year study of a hundred or so college teachers in a wide variety of fields and universities.  It comes to the conclusion that it is not what teachers do, it’s what they understand.

Techniques and stuff (like lesson plans) matter less than the special way teachers view their subject and value human learning (or not!!). The best teachers, according to the study, know their subjects well, and also know how to “engage and challenge students and to provoke impassioned responses”.

From the book:

Most of all, they believe two things fervently: that teaching matters and that students can learn.

It highlights the research that got me launched in thinking about teaching and learning in the early 1990′s: a bunch of physicists involved in the area of educational research.  These guys introduced me to Vygotsky.  It’s been an interesting mix actually: the kind of talk given by a physics lecturer on educational theory is quite different to other talks.  Google “PER physics” (PER=Physics Education Research).  They also have a different set of mental models, and some interesting (odd??) juxtapositions of ideas.

It is a short, well constructed, evidence based (is that the right term??) inspiring little book.  I’ve wondered about a reading group around this book – or something.  My first thought was something international through the POD group.  Maybe.  Watch this space.