This page is located at: http://www.ncsall.net/?id=472
A Productive Partnership
Richard J. Murnane and Bob Bickerton
Rarely do researchers and practitioners engage in serious dialogue. NCSALL enticed us, an
economist and an adult educator, out of conventional patterns when it requested that we
jointly write an article on a research study that raised questions about the value of the
GED credential. This article describes the results of our discussions.
In 1993, the University of Chicago economist James Heckman, working with graduate student Stephen Cameron, published a paper entitled "The Nonequivalence of High School Equivalents." The main message was that young male recipients of the GED credential did not fare as well in the labor market as young males with conventional high school diplomas. The findings were widely reported in both the broadcast and print media.1
Our reactions to this article were quite
different.
Murnane: I found the
research compelling, in part because Heckman is one of the
country's most widely-respected laboreconomists and the research seemed carefully done,
and in part because the central finding replicated a pattern reported by David Passmore in
the September 1987 American Council on Education GED Research Brief. Indeed, I was puzzled
by the acrimonious interchanges between Heckman and representatives of the American
Council on Education (ACE) since the main finding of the Cameron-Heckman study
did not
differ from a finding reported six years earlier in an ACE publication.
Bickerton: I was troubled initially by the articles on the Cameron-Heckman study
because they had the potential to diminish the value of the GED and appeared to contradict
my own observations and experiences. As someone involved in ABE policy, I needed to know
more. I must admit that I was particularly interested in the study's weaknesses, which I
felt were confirmed by the ACE/GED Testing Service argument that the study was flawed
because it concentrated on young males, disregarding the reality that most GED recipients
receive this credential later than their high school graduate peers receive theirs. When
subsequent interviews with the study's authors continued to focus on decontextualized
statements such as "no significant difference between the earnings of GED recipients
and drop outs" without serious attention to its limitations, I presumed that this
was, in fact, an unbalanced attack on a credential important to millions of adults as
their second chance opportunity to pursue their dreams. This made me even less receptive
to the study, disconnecting me from findings which could help inform the improvement of
our policies and programs.
Asking the Right Questions
When adult educators read research on the GED, the questions they tend to have in their minds are: Do individuals who lack a high school diploma benefit from obtaining a GED? In what ways economically, socially, personally? Is the answer different for men than for women? For immigrants than for native born Americans? What challenges must GED recipients overcome to obtain benefits from the credential?
These questions are not Cameron and Heckman's primary focus. Instead, their attention, and the focus of the media coverage, was on a different question: Do young male GED recipients fare as well in the labor market as young males with conventional high school diplomas? As we worked on this article, we came to understand that the media's lack of interest in the differences among these questions contributed to some of the controversy generated by the Cameron-Heckman paper, which concluded by suggesting that the GED system was no longer of use.
Comparing GED and Grads
We came to agree that Cameron-Heckman's research comparing labor market outcomes of young male GED holders and young male HS diploma holders has important strengths. The outcome measures, hourly wages and annual earnings at ages 25 and 28, are clearly important. The research design is solid: Cameron-Heckman compared labor market outcomes for males with different educational credentials after using statistical techniques to account for differences in family backgrounds. We came to agree that Cameron and Heckman are right: young males who obtain a GED prior to the age of 25 do not fare as well in the labor market at ages 25 and 28 as males with conventional high school diplomas do.
At the same time, we came to agree that, while the media paid little attention to the limits of the Cameron-Heckman sample, they are important to keep in mind. Cameron-Heckman provide no information on the value of the GED credential to females. Nor did the study provide information on the value of the GED for adult immigrants, who constitute a large part of the clientele for adult education, or for adults who obtain a GED after the age of 24, about 36 percent of recipients nationally.
Economic Benefits?
An important question to adult educators and to us is whether school dropouts gain economic benefit from obtaining a GED. A reliable answer requires a comparison of the labor market outcomes of GED recipients with the labor market outcomes GED recipients would have had, had they not obtained the credential. Of course, by definition, this counter-factual comparison group does not exist. The conventional research strategy is to use dropouts who do not get a GED (permanent dropouts) as the comparison group. This strategy has some flaws because, on average, GED recipients completed more years of schooling before dropping out than did permanent dropouts and they had higher grades and test scores while in school. As a result, the GED recipients probably would have fared better in the labor market even without the GED credential than permanent dropouts. Researchers typically use statistical methods to control for differences between GED recipients and permanent dropouts to isolate a GED effect, although this can be difficult and may not be completely satisfactory.
Cameron and Heckman used the best of conventional methods to explore whether young male dropouts benefit from obtaining a GED. Contrary to the summaries provided in most media reports, they viewed their results as inconclusive, writing the...data strongly reject the hypothesis that GED recipients are the labor market equals of high school graduates. The same data do not reject the hypothesis that high school dropouts and GED recipients are indistinguishable. A closer look at the evidence indicates, however, that GED recipients lie between dropouts and graduates in their economic standing....
So, what does it mean when the study concludes, "there is no significant difference between the earnings of high school dropouts and GED recipients?" The question is especially important in light of Cameron and Heckman's finding that, in the sample they investigated, the wages of young male GED recipients were 3 to 11.5 percent higher, on average, than those of dropouts without this credential. While these differences are modest relative to the difference between the average wages of GED recipients and conventional high school graduates, they could be large enough to be important to school dropouts. The practice of sound research requires, however, that we explore whether the differences between the average wages of GED recipients and permanent dropouts could have surfaced by "chance." In this study, we find that there is so much variation among the wages of GED recipients and among the wages of permanent dropouts that we cannot reject the possibility that these higher wages are just a chance occurrence. The inability to reject the hypothesis that the differences could have arisen by chance is what the statement "not statistically different from zero" means. Keep in mind, however, that tests of "statistical significance" are highly dependent on sample size, and the size of the Cameron and Heckman sample is modest (645 males). Had the sample size been larger, the standard errors associated with these point estimates would have been smaller, and it is possible that the difference between the average wages of GED recipients and those of permanent dropouts would have been statistically significant.
Lessons from the Dialogue
Our discussions taught us a number of lessons that we believe can help researchers design more valuable research and help adult educators make sense of and learn from research.
Read the research report, not only the media coverage.
Our discussions of the Cameron-Heckman paper convinced us that the media coverage was incomplete, and in many cases misleading. A careful reading of the paper also reveals that some findings are open to alternative interpretations. For example, Cameron and Heckman report that GED recipients are more likely to participate in postsecondary education than permanent dropouts are, and that college pays off for them. Cameron and Heckman interpret this finding as indicating that most of the positive effects of GED acquisition on labor market outcomes stem from the postsecondary education, not from the GED credential itself. Many adult educators would agree, but would emphasize that improving access to postsecondary education is a critical benefit of the GED.
Clarify what questions are being addressed.
Most media reports of the Cameron-Heckman paper did not distinguish between the two very different questions the paper addressed: how do young male GED recipients fare as compared to high school graduates and how do they compare to dropouts who do not complete the GED.
Identify the comparison group.
Some studies simply compare labor market outcomes for GED recipients before they obtain the credential with outcomes after receipt. This is unsatisfactory because labor market outcomes typically improve with age irrespective of whether one obtains an educational credential or not. The relevant question is whether the improvement with age is greater for GED recipients than it would have been had these dropouts not obtained the credential. To explore this question, a comparison group is needed to provide an estimate of what labor market outcomes would have been had the GED recipients not obtained the credential. It is important to identify the comparison group and ask whether it seems to be a close match for what GED recipients would have been like had they not obtained the credential.
Identify the sample.
In general, the track record of programs to improve labor market outcomes for disadvantaged young males is disappointing. The track record of programs to help disadvantaged females is somewhat more optimistic. One lesson here is the importance of identifying the sample used in a particular study and recognizing that it is inappropriate to generalize results beyond the group from which the sample is drawn.
Pay attention to the time period that was studied.
The Cameron-Heckman study used labor market data from the mid 1980s, a period when the transformation of the American economy was in full bloom. A part of this transformation was a decline in labor market opportunities for less skilled workers, including permanent dropouts, GED recipients, and conventional high school graduates. One indication of this transformation is the 25 percent decline since 1979 in the real earnings of 30 year old male high school graduates. This decline has prompted K-12 educators to push for higher educational standards and for examinations that would certify that high school graduates are prepared to compete for good jobs in an economy in which problem solving skills, communication skills, and the ability to use computers are increasingly valued.
We agree that adult educators need to respond to the decline in the earnings of school dropouts in a similar fashion by considering whether changes in the content and format of our curricula and the GED examinations would provide students and employers with better evidence that recipients possess the skills needed in high wage jobs. Can we be content with a GED science test that is based primarily on reading comprehension? Is a testing format in which the answer to each question is one of five pre-defined choices appropriate in a world in which the good jobs require problem diagnosis and significant communication skills? The introduction of the essay section on the GED Writing Skills Test in 1988 was one promising response to the changing economy. Additional responses will also be needed as the skills needed to obtain good jobs - jobs that pay enough to support children continue to grow.
New Research
The central point of agreement that we reached after extensive discussion is that more research is needed on the role the GED plays in improving labor market outcomes for different groups in a changing economy, and on the impact of the GED in other arenas. With support from NCSALL, one of us (Murnane) is in the midst of a large scale study with colleagues at Harvard to develop better methods of defining an appropriate comparison group against which to compare the labor force outcomes of GED recipients.
By writing this article, we have learned that the knowledge and perspectives that each of us brings to discussions of the GED are relevant to the other person's work. We plan to explore how to structure future dialogues that will both improve the quality of the work on the GED, and make it of the greatest use in improving the quality of adult education in the United States. And it is abundantly clear to both of us that policy and practice will be greatly enhanced the more frequently practitioners, researchers, and policy leaders come together to create a more complete, accurate, and useful body of knowledge upon which to build these critically important services.
Endnotes
1 S. Cameron and J. Heckman, "The Nonequivalence of High School Equivalents," Journal of Labor Economics, 11 (June 1993b) 1: 1-47.