In October
2005, the Policy and Program Studies Service of the United States Department of
Education released a 148-page "review" of research on single-sex
education. This review was commissioned by the Department of Education
and was performed, under contract, by five individuals at the American
Institutes for Research (
The review is
not friendly to single-sex education, which is not surprising, considering
what we know about RMC Corporation, but its not well-researched or
well-executed, which is a bit surprising, considering that the
Department of Education paid real money to have five people write this thing.
I (Dr. Sax)
sent my criticism of this review to the five authors, since I thought it was
only fair for them to have a chance to respond to my criticism before I posted
it online. Those authors were courteous enough to respond. Their
response follows below, after my criticism.
The review
is entitled "Single-Sex vs. Coeducational Schooling: A Systematic
Review." You can read a summary of the report, and link to the full text
in PDF format at this link: http://www.ed.gov/rschstat/eval/other/single-sex/index.html
Criticism #1: The "systematic quantitative review" is neither
systematic nor quantitative.
The authors
describe their report as a "systematic quantitative review" (page 11).
They try to make their review appear quantitative by listing each study they
reviewed as either "pro-SS, pro-CE, or null" (in favor of single-sex education,
in favor of coeducation, or favoring neither). They then add up the
number of studies pro-SS, the number of studies pro-CE, and the number of
studies which are null. However, they do not take into consideration the
number of students in each study.
For
example, the 2002 report by the National Foundation for Educational Research
showed a dramatic beneficial effect of single-sex education in broadening
educational horizons for girls: girls who attended single-sex schools
were much more likely to study advanced math and science than were girls of
comparable ability who attended coed schools. This study involved 369,341
pupils from 2,954 schools, and was published as a 108-page monograph. But
for the authors of the Department of Education report, it gets just one check
mark in the pro-SS column, on page 51 of their report.
The authors
devote the entire previous page, page 50, to an unpublished, unreviewed study – a presentation at a conference of
university administrators in
There is no
weighting whatsoever for the fact that the published study favoring single-sex
education involved 369,341 students at 2,954 schools, in hundreds of different
cities and towns, while the study purportedly favoring coeducation was an
unpublished presentation at an obscure conference, involving fewer than 200 students
at just two schools in one community. As far as these authors were
concerned, it was a wash. Quantitatively speaking.
One study "pro", one study "con."
This sort
of writing is not systematic. It is not quantitative. It is not
scientific. It is poorly executed.
The
Department of Education should demand their money back!
Criticism #2: The "exhaustive, systematic review" is neither
exhaustive nor systematic.
The five
authors of this 148-page report assert, three times, that they conducted "an
exhaustive search" of the published scholarly literature. In other words,
they are telling us that they included all relevant studies. But they did
not. For example: The authors pose the question of whether students
at single-sex schools are more or less likely to develop eating disorders,
compared with students at coed schools. On this last question, the
authors "exhaustive search" turned up just one study, according to which girls
at single-sex schools were more likely to have eating disorders than girls at
coed schools. The authors say that they
used the PsychINFO database as one of their principal
sources. However, a quick search of the PsychINFO
database reveals not one, not two, but four studies which meet the
authors quantitative criteria. You will find the four studies listed (in PsychINFO format!) at www.singlesexschools.org/eatingdisorders.htm.
Two of those studies found that girls at coed schools wanted to
look thin because they thought that's the way the boys wanted them to
look. Girls at single-sex schools were much less concerned with what the
boys wanted. None of this appears in the Department of Education
report.
The
authors excuse (see the full text of their response, below) is that they
were not able to get the full text of these other studies, so these other
studies were not included. That excuse is not persuasive. The full
text of these articles is readily available from services such as Infotrieve.com.
Another
example: the authors list six studies which examined the effect of school
format, coed vs. single-sex, on self-esteem. However, the best-designed
of these studies, the famous study by Jacqueline Granleese
and Stephen Joseph, which I described in the closing chapter of , is not even mentioned anywhere in the report. As you may
recall, Granleese and Joseph found that for girls at
coed schools, personal appearance, i.e. whether or not the girl believed she
was pretty, was by far the most important determinant of self-esteem.
For girls at coed schools, if you are pretty, life is great. But if your
appearance does not qualify you as one of the cute girls, then
life is not so great. At coed schools, for girls, the focus is on how you look. For girls
at single-sex schools, though, appearance was not a key determinant of
self-esteem. The focus at girls schools is more on who you are rather
than on how you look.
This study
is never mentioned in the Department of Education report, even though the study
fulfills all the criteria for inclusion.
Criticism #3: The "exhaustive, systematic review" is not a
review, but merely a bibliography.
The most
important failing of this expensive study has nothing to do with the fact that
the authors missed lots of important studies, despite their "exhaustive
search." The most important failure was the failure to address the key
question raised by any review of this literature, namely: WHY are some
schools successful with this format, while others fail? The authors list
studies which show that boys did better at single-sex schools than at coed
schools; they list studies showing that boys did worse at single-sex schools;
they list studies showing that school format had no effect. The authors
never address the question: why? Why does single-sex education lead
to tremendous gains at some schools, and disastrous outcomes at others?
That is why
I consider this report merely an annotated bibliography, not a review. A
review includes some DISCUSSION of the variation in the results, and some
attempt to consider various HYPOTHESES which might account for the
variation. This report contains none of those elements, merely an
annotated listing of articles.
Criticism #4: The authors of the study give no consideration
whatsoever to the possibility that GIRLS
If teachers
do not understand that girls and boys learn differently, then the single-sex
format is not a recipe for success: in fact it may lead to disaster.
please consider two recent examples
from American public schools. On
On the very
same day,
The authors
of the Department of Education study would score the
The authors
of the Department of Education study might suggest that these programs are more
likely to succeed in low-income, underprivileged neighborhoods serving
primarily Black or Hispanic students. That explanation will not work
here. Reynolds Middle School has a higher proportion of students
receiving free or subsidized student lunches than the Woodward Avenue has, and
the Reynolds school has an almost 100% minority population compared with less
than 50% at Woodward Avenue. Yet the programs at Reynolds failed, while
the program at
In fact,
the explanation is simple and straightforward. Teachers at
Right
there, Ms. Rodkey has identified a key variable which the authors of the
Department of Education study never address: namely, the teachers' attitude
toward the format. Recall that in the
Girls
and boys learn differently. That's the key point, of which the authors of the Department
of Education study show not the slightest awareness. Indeed, their jargon
is redolent of the 1970's sociologists who would dismiss such as a claim as
“essentialist.” Studying single-sex education without any awareness of
gender differences in learning is like studying the ups and downs of the Dow
Jones stock index without any knowledge of what stocks are. You can make
all sorts of charts and computations, but in reality you don't have any idea
what you're talking about.
That's why
our Association generally, and I personally, have serious reservations about
any study which compares "single-sex schools" as a category with
"coed schools" as a category. Just putting girls in one
building and boys in another is not a guarantee of success. If teachers
don't have appropriate preparation, and knowledge of best practices for
single-gender education, the single-gender format has no pedagogical advantages
over the coed format, and may in fact result in educational shipwreck. I
made this point at some length in my essay for Education Week published earlier this year. You can
read that essay at this link:
http://www.singlesexschools.org/edweek.html
That
concludes my criticism of the Department of Education report – at least for
now.
phone:
301 972 7600
fax:
301 972 8006
P.
S. As noted
above, I sent an earlier version of this e-mail to the authors of this
report. The earlier version did not include criticism #1.
Here
follows the response from the authors of the report, complete
and unedited:
Dr. Sax,
Thanks again for allowing us to
reply to your criticisms about our review. You raise some very relevant
and good points about some potential omissions to the review, as well as some
more general criticisms. We have addressed these in the comments below.
1) On your point about studies concerning student eating disorders, you
obviously found additional studies than the single one that was included in our
review. There was a good reason for that. We also found those
additional three studies in Phase I of the review (see pages 3 and 4 in our
review). However, they appeared in journals that we were not able to
locate (as we needed to review the full text of the articles, rather than merely
the abstracts). Our efforts involved a number of steps: 1) search for the
full text of the article in PsycINFO, 2) search for
the article using our organizational library resources (which involved
contacting the journals directly about purchasing the article) 3) search for
the article at local universities, and 4) attempting to contact the authors
directly (if we were able to locate their contact information). In all
three cases, none of the steps resulted in obtaining the full texts of the
articles. Therefore, they could not be included in the review.
2) On your other example of studies concerning student self-esteem, we did
miss the Granleese and Joseph article that you
mention. This article did not turn up based on our searches of the research
article databases (see page 3).
3) Your most significant criticism of our review is that it does not deal
at all with the “why” question – why single sex schools would ever work better
and under what conditions they would work better. That is correct; the report
on the website does not cover “why” issues, as it was beyond the scope of the
review. We actually authored a separate manuscript which addresses the
potential reasons why single sex schools might work better or worse than coeducational
schools, including the argument that boys and girls have different learning
styles and are best taught in environments geared to their unique styles.
Although this paper was not put on the USED website, it is currently under
review at a peer-reviewed journal.
Again, we appreciate the opportunity
to reply to your criticisms and are happy that this important issue is
receiving its due attention.
Sincerely,
----