What Works? Under What Conditions? And Who Really Cares?

© 2003 Edward G. Rozycki, Ed. D.

One looks back with appreciation to the brilliant teachers,
but with gratitude to those who touched our human feeling.
-- Carl Gustav Jung

RETURN
edited 9/3/11

What Works? To What End?

Every experienced teacher has a "toolbox of approaches" or a "bag of tricks." These tend to be methods expected work in certain circumstances and not in others, although what those circumstances are is seldom spelled out. People who claim to have a methodology, by contrast, are insinuating either that their particular tricks

a. work generally over varied populations of students; or

b. are applicable to identified particular populations.

Those who typically offer school districts panaceac methodologies [1] are angling for big bucks. The kindest generalization I can offer as to their value -- having observed the field for almost forty years -- is that such "methodologies" at best rest on weak science, sell primarily to support wishful thinking and not infrequently amount to downright fraud. [2]

This is not to say that there is some "good old fashioned" method that works best. A popular newsmagazine recently offered the opinions of another befuddled educator: what works best is a good teacher with a chalkboard and a class of willing students. This is not even an item in a bag of teacher tricks; it is rhetorical prestidigitation: a restatement of the desired end is taken to be a specification of means. It is like saying that World Hunger will be solved by getting food in empty bellies; or, that health will be achieved by eradicating sickness. Such buffoonery is not uncommon in many areas of our public discourse.

U. S. public education is conducted relatively openly in our democratic, pluralistic society. Let me specify what these terms mean.  In educational discussion, our society is thought to be pluralistic to the extent that values are not shared. (Our public schools, strangely enough, promulgate a mindless celebration of the very pluralism that undermines their rational governance.)

Our society is democratic, not so much in political governance, as in public rhetoric: all opinions, from the least informed, most prejudiced animadversions -- according to the fashions of "political correctness" - to carefully weighed judgments are given equal hearing; and, unfortunately, too often, equal weight.

Our public discourse, even among technically focused educators, tends towards a vacuous rhetoric which displaces a suspiciously elitist linguistic precision. Thus, we Americans, lacking the means to even identify, much less achieve, ill-advised or unclear school goals, persist in trying to fix schools that either aren't broken, or which can't be fixed. [3]

What Works? Can Efficiency Be Achieved?

Many teachers are superb technicians and get results with all kinds of children. The problem is that it is difficult to say explicitly just what it is they do. This is why many feel teaching is more an art than a science. Unfortunately, many important causal factors in pedagogy await discovery. It is difficult to generalize from one successful classroom the circumstances that led to the particular success observed. This tends to make people believe that good pedagogy is a mysterious matter, and to exaggerate charisma. In the popular media, good teachers are evangelists, magicians or entertainers.

Henry D. Levin [4] has proposed that the following conditions are assumed by the pursuit of productivity in schools:

-- Knowledge of the technical production process.

-- Substantial manager control over the input mix.

-- A basically competitive environment

-- Managerial knowledge of input and output prices

-- The goal of maximizing output (not necessarily just profits)

-- Clear signals of success or failure (profits, losses, market shares, etc)

Can these conditions be met in public schools?  Let’s examine what they mean more closely.

A Basically Competitive Environment. This requires that incentives exist for experimenting to find efficient means of production.  Outcomes should not be indifferent to technique.  This is the point of proposals to provide educational vouchers so parents can select schools they believe are more effective.  But it is not clear what would count as more efficient technique or that parents would be the best judges.  Is a technique more efficient if it brings a few students to higher achievement, or if it brings more students up to a lower common standard? At whose cost should someone else's achievement be gained?

Managerial Knowledge of Input and Output Prices.  What this means is that some standard method of gauging costs and benefits is needed so that a teacher could decide among different techniques on the basis of those costs and benefits.  Not only does such a standard method not exist, but both law and tradition militate against this practice in the school.  For example, "teaching to the middle" is a practice teachers engage in because it shows greatest gain for a given amount of effort.  But special education laws and moral considerations often direct teachers to expend effort on students whose achievement will cost many times that of a normal student -- measured in teacher hours.

The Goal of Maximizing Output.  This goal is not a priority in schools.  Both moral and legal restraints prevent its realization -- not to mention a lack of consensus on what would count as a measure of input and output.  Teaching efficiency, for example, is constrained by considerations of morality and status. [5]   Let's look more closely at this.

Consider two examples of motivational devices of ancient though ill-repute: the torture-chamber and the brothel.  No one doubts that certain kinds of behavior could be motivated by the use of torture or the promise of sex.  No doubt standard learning outcomes could be enhanced through these means.  Why are they not used?  Many people would find them morally objectionable, particularly where children are involved.

Teachers are careful not to single out or group students by race, sex, SES or other characteristics possibly pertinent to school achievement. Although such grouping might increase the efficiency of instruction, other outcomes are feared, such as stereotyping. In fact, there are legal barriers to grouping students on the basis of these characteristics.

Not only means, but ends come under moral restriction. Who would teach any of the following in an American public school: cheating at cards, safecracking, mugging, or begging? Why are highly useful things -- economically speaking -- not taught in the public schools? For example, slaughtering pigs, fixing toilets, or handicapping horse races?  Why not teach mumbly-peg, tree-climbing, half-ball or solitaire? Because some of these are thought to be inappropriate to the school, out-of-place because they are too crude, too low-class or too trivial.

Clear Signals of Success or Failure. For the reasons mentioned above, there are no such signals that enjoy wide acceptance.  Good grades or bad grades, for example, are only a sign of success or failure if you trust the process which generates them.  The fact is that parents, by and large, trust this process, to judge from Gallup Poll results over the years. However, this trust gives no advantage to school people when the public believes that situations which are likely beyond the power of the school to deal with can be treated by a mere addition to the curriculum, e.g. character education.

What Works, 1986? [6] Under What Conditions?

There are specific things that parents could do to increase the probability of their child's success in school.

Children who are told stories by their parents are motivated to read.

Children who are read to by their parents do better in school.

Children who are encouraged by their parents to do leisure reading have larger vocabularies and greater reading fluency.

Children whose parents encourage the counting of everyday objects are more likely to succeed at higher level mathematical tasks.

Children who are engaged in thoughtful discussions on anything they have an interest in are likely to have higher academic achievement.

Children who are encouraged by their parents to draw and "write" develop crucial language skills.

But from a systemic point of view where background conditions and cause and effect relationships are carefully examined the question becomes, "What conditions are likely necessary for parents to follow these practices?"  We can think of several:

The parents know what the research indicates.

The parents feel that doing better in school will ultimately mean something for their children.

The parents have the time and energy.

The parents believe that it is appropriate to their role.  (For example, in certain cultures, fathers might not feel that reading to little children is manly.)

The parents believe that these suggestions do not violate the proper lines of authority between parent and child.  (For example, some subcultures do not encourage the discussion of ideas with children.)

The parents are able to afford materials that could be "wasted" by children in practicing drawing and writing.

These conditions illustrate the many factors that are at work that permit teaching methodologies to work. It is stupid to treat the outcomes of research as a cookbook for success in school.

What Works, 2002? [7]   And the supporting conditions?

If someone claims to have a method, call it method M, here are some questions to pose to make sure we are not getting useless generalizations:

a. What exactly does M seem to have accomplished in the past?

b. How is M distinguished from other competing methods?  By procedure?  By cost?

c. With what kinds of and how many students has M worked?

d. What kinds of teachers used M?

e. Was the research design under which the efficacy of M was tested a good one? [8]

Let's look at some recent candidates proposed as effective and inquire as to the assumed background conditions and possible cause-effect relationships:

Proposal 1. Parental Involvement and Partnerships Schools should encourage parents and develop cooperative partnerships with the home to stimulate their children's intellectual development.

What kinds of parents are likely to get involved with cooperative partnerships? Perhaps it is their personal characteristics rather than the school partnership that have the greater effect on their children's intellectual development.

2. Homework and Feedback  Homework that is graded, commented upon, and discussed holds more learning value for students.

But homework must be done for this to occur. Perhaps the student's just working at it is the causal factor; the class discussion might merely create a desire in the student not to be embarrassed for not having done it.

3. Goal Setting and Time on Task: Time on task and proper goal setting should be reinforced by parents and teachers.

Students who are prepared to spend time on task and set proper goals may already be high achievers quite independently of their being able to articulate "proper goals."

4. Direct Teaching: Good old-fashioned teaching is still effective.

Which "old-fashioned teaching" is good? For whom is it effective? And when? Parents often prefer their child be given good old-fashioned verb conjugations in language classes than have them be taught by any of the newer methods with scientific pedigree.

5. Cooperative Learning:  By working in small groups, students learn teamwork, how to plan, monitor, and evaluate their activities as well as receive and give constructive criticism.

One must learn to work in small groups. It is just as likely that having learned teamwork outside of school -- say, by game-playing -- they then bring these skills into school.

The Basic Rules of Public School Hyperbole

Because public educators must deal not only with their students and colleagues, but also with parents, reporters and assorted busy-bodies, in other words, with the wishful thinking, the ill-informed and the near-delusional, they, the educators tend to adapt that maxim so eloquently articulated by Thumper's mother in the movie, Bambi: "If you can't say something nice, don't say it at all." In the name of motivation, tact or good publicity, we become casual liars.

There is, in addition, an aspect of advertising culture that has been picked up by the hyperbolists of American public education: the reversal rule, parts A and B

The Reversal Rule, part A is this: if it's true but unpleasant, treat it as false.

The Reversal Rule, part B: if it's false but pleasant, say it anyway.

Thus rather than recognize the ancient maxim, "Impossibility negates obligation," we are encumbered with "All children can learn!" or "No child left behind!" The santimoniousness of this blather is supposed to make us forget, perhaps, that the concerned public servants who have foisted off overexacting special education legislation on the schools are the very ones who have reneged on their promises for adequate resources.

What Can and Ought We Do?

Informal, uncritical discussion of methodology is basically a ruse to distract educators from important underlying issues: What goals ought we pursue? At what cost and to whom? How do we balance the many, often conflicting obligations we have as educators to our pupils, both individually and as a group; to our community, to ourselves, etc.

We can begin by recognizing that although the basic rules of public school hyperbole may be politically necessary, their benefits extend not much further. Unless we keep a clear mind, such rules threaten to be come the instruments of our own self-delusion. There are some basic, unpleasant truths we cannot dodge:

a. Any standardized curriculum, methodology or treatment will tend to disadvantage some students and advantage others.

b. What is at present legal, may be wrong.

c. What is scientifically established, efficient and humane may not be what parents want the schools to teach their children.,

d. Professional knowledge is anything but absolutely certain: an incautious educator is probably a charlatan.

e. Teaching and administration in public schools tends to be underappreciated, often thankless and stressful.

 f. Saying, "No!" can have effects which are neither illegal, immoral nor fattening.

Two final steps in our methodology complete one's professional development: stop whining; and, grow up.