printable version of page Printer-friendly page

Focus On Basics

Volume 2, Issue C ::: September 1998

A Story of Improvement

by Jane Cody, James Ford, and Kathleen Hayward
An honest look told this literacy program they were not as good as they might be. It was time to listen to stakeholders and institute a continuous improvement process.

"The art of progress is to preserve order amid change and to preserve change amid order "
- Alfred North Whitehead

Looking in the mirror, really taking a long, hard look, can be unnerving, even depressing. We see the blemishes that we hide with makeup. We see the bald spots that we cover by creatively combing our hair. We see the ten extra pounds that age has brought. Most of us avoid the problem by not looking, or we take a quick glance to make sure we are properly buttoned and zipped and ready to appear in public. Perhaps we make a silent resolution about dieting or exercising, but that's about it. After all, we're not perfect.

This is pretty normal human behavior and it carries over into our professional lives. After operating an adult literacy program for many years, it is easy to avoid taking a long, hard look in the mirror. Students come, money comes, programs expand, people say "well done," everything seems pretty good: not perfect, but pretty good.

This is the story of an adult literacy program that decided that merely glancing in the mirror was no longer enough. It is a story of shock: of discovering a lack of clearly defined standards and the tools to measure progress toward them. It is a story of hope, in finding a process to identify program gaps and narrow them. It is also a story of improvement, through the systematic use of stakeholder input, hard data, and detailed analysis. It is a story, not about perfection, but about getting better.

Our Organization

Adult and family literacy programs in Knox County, Tennessee, are provided by a unique public/private partnership. The public member is the Knox County School System and, specifically, its Adult Basic Education Department (ABED). The private member is a nonprofit corporation called Friends of Literacy, Inc. (FOL), which was formed in 1991 to supplement - by way of money and volunteers - the resources that ABED had for literacy programs. Through the balance of this article, "we" refers to this FOL/ABED Partnership, to the program it produces, and to its combined leadership.

Most knowledgeable observers would have described this as a successful partnership with high-caliber programs. Funding grew each year, students provided ample anecdotal evidence that we made a difference in their lives, and we had achieved some national recognition. We considered ourselves anything but static. We had changed our curriculum, improved our volunteer training, and added more paid teachers. Our program had grown to approximately 400 students, 15 paid staff, and 40 volunteers. We were doing something right.

Forces for Change

In 1996, FOL had its fifth birthday. Its budget had grown from $5,000 in 1991 to more than $150,000 in fiscal year 96-'97. It had evolved from an organization that provided money and people to an adult literacy program operated by ABED to one that was also the primary management and fiscal agent for its own family literacy program. It was time to take a look at where FOL, the Adult Literacy Program, and the partnership were going. As luck would have it, new members had just been added to the FOL board of directors. These members had not grown up with the organization and, consequently, viewed it from a different perspective. One new member had expertise in strategic planning. A committee composed of FOL board members and program staff reviewed and clarified the vision, mission, and values of FOL.

In addition, the FOL/ABED partnership had been selected to participate in a national project entitled "What Works Literacy Partnership" (WWLP). WWLP sparked our thinking, in a more detailed way, about how we could document the results we thought we were achieving in improving the basic academic skills of our students. We started to concentrate on how could we prove that our program really did what we said it did.

Dovetailing with WWLP was another national initiative entitled "Equipped for the Future" (EFF).The FOL/ABED partnership was selected to be one of several pilot-test sites for EFF. Like WWLP, one aspect of EFF focused on the question of how we measure results.

Two other critical pieces fell into place during the summer of 1997. First, to help us with the WWLP project, we hired a new staff member who just happened to be a Certified Quality Examiner for the State of Tennessee (more about this below). Second, FOL received a generous gift to purchase new hardware and a software system to manage data about students, volunteers, and donors. With all this going on, we had both the courage to look in the mirror and some ideas about how to take a systematic approach to dealing with what we saw there.

New Approach

Our new staff member introduced us to the Malcolm Baldridge Educational Criteria for Performance Excellence (see Figure 1). These criteria take the total quality management principles originally used in the private sector and apply them to educational institutions. They provide an orderly approach to continually improving the operations and results of an educational enterprise. This was the systematic approach we would use.

The starting point was to identify the needs of the customers: in this case, our students. FOL/ABED program leadership instituted monthly "Town Meetings" to ask students what they wanted, needed, and expected from our adult literacy program. Approximately 75 students attended each of the meetings. We got a wide range of responses, from better parking to more computer time and more volunteers for individual tutoring. Next, we asked teachers and volunteers, who are key stakeholders in the program, what they expected and how we could all work together to improve things. Better training and more support from the leadership were two answers. This group also developed its own vision, mission, and value statements for both adult and family literacy program components. These built on similar work completed earlier by FOL and the Knox County Schools, but were the work of these stakeholders, and were something they could commit to as their own. The FOL board also brainstormed its role and the expectations of and for each board member. Some consistent themes were emerging, one of which was better communication among the stakeholder groups: students, teachers, volunteers, and board members.

We wanted to use this information to develop quality standards for our organization and program. We were fortunate: adult literacy educators in both Tennessee and New York had already adopted "Indicators of Program Quality" that fit nicely into the Baldridge Criteria and our own vision, mission, and values. For example, Quality Indicators One and Two of the New York program stress the involvement of all stakeholders in the development of program philosophy and direction. Quality Indicators Three and Five underscore the importance of measuring student improvements in reading, writing, speaking, and problem solving, and highlight the goal of enabling students to be better workers, parents, and citizens. These Indicators were consistent with our approach in the EFF project. Indicators Seven and Eight set benchmarks for the recruitment, orientation, placement, and retention of students. We felt comfortable with all 13 Quality Indicators and adopted them as standards by which to measure our own work.

Measuring progress towards benchmarks and ultimate program results are critical concepts in the Baldridge Criteria. We realized that we must start "managing by facts," rather than by assumptions or guesswork. We set about the task of collecting and analyzing data in several ways. First, program management went back into the classroom to see what actually happened there on a daily basis. They resumed the role of teacher to see, firsthand, the problems of taking our curriculum from training manual to actual delivery. They also observed other teachers to determine whether the material covered during in-service training was actually being incorporated. These were eye-opening experiences. What we thought was happening in the classroom was not always occurring. Some of the curriculum was unwieldy or too complex. It was not always well integrated between teachers in different classrooms. In our family literacy program, for example, the curriculum used by those teaching the parents was not well linked with that used to teach the children. Teachers provided input to help redesign curriculum areas that were not working.

We needed better data on actual student performance, so we changed to a more accurate instrument that provides diagnostic information to the student and the teacher. We tested adult learners using the Test of Adult Basic Education (TABE) when they first enrolled in our program, and again after completing 100 hours of classroom instruction. We collected, analyzed, and fed data back to both teachers and students. We use this information to develop specific, individualized action plans for improvement by each adult learner. At last we had reliable, hard data to show us whether our students were making academic progress.

As we were gathering data, a "think tank" of program managers, teachers, and FOL board members began to identify the "vital few" core drivers of our partnership and literacy programs. Core drivers are what we have to do well to meet the indicators of a quality program. They included student recruitment and retention, volunteer management, financial management (raising and spending money), and bringing our new data system up to speed. The group examined how we operated in each of these areas, set sights on a higher standard for future performance, and developed detailed action plans for improvement. For example, we knew our data system was critical to managing by fact. Without a better system, we could not record and analyze data on student test results. Neither could we keep track of who had given us money, when, how much, and for what purpose. We solicited input from everyone who had to use the data system, asking, "What are the problems with current operations and how could they be improved?" We selected a point person to deal with the software company in redesigning some elements of the system, based upon this input, and we established a timetable for improvement.

Shock, Hope, Improvement

The Baldridge criteria require an organization to slow down and rebuild with a focus towards customer and program results that matter. The criteria link and create an alignment between all internal operations, organization leadership, systems, and processes as well as sound literacy standards. When we first began our critical look, many of us confidently believed that, on a scale of one to ten, with ten being the best, we would be about an eight. Wrong. Systematic feedback, data collection, and analysis, as described above, showed some real gaps in our operations. For example, teachers told us that the continuous enrollment of new students on whatever day they happened to show up was a real barrier to success. It required teachers to interrupt classes to assess new students and try to bring them up to speed. Students told us that it was intimidating to be assigned to a classroom without any idea of what to expect upon arrival. The solution, devised with input from the stakeholders, was a new class, called Learning Skills, in which each new student is enrolled for a month before assignment to a regular class. Our TABE test results indicate that students who attended the Learning Skills class are progressing more quickly than those who were enrolled in our program prior to development of this class. In addition, since students graduate from Learning Skills at the end of the month, teachers in regular classes enroll new students only once a month and have benchmarks for them before they arrive. See the box for a description of this class.

Teacher training was another area that needed to be addressed. We found that teachers were not incorporating in-service material in their classrooms. We asked them for help in revising classroom curriculum, and we changed our training focus from putting on a workshop' to meeting the needs of the staff. In addition, our EFF implementation allowed us to bring together teachers who worked in different parts of our program. Lack of funding for teacher planning time had been a significant barrier to this in the past. Adult literacy teachers linked up with family literacy teachers. Staff began exchanging ideas and method; they brainstormed to solve problems. When asked to evaluate this year upon its completion in June,1998, teachers were significantly more positive about the program, and their role in it, than they had been in the past.

We also saw another significant change during the year. We had tried several times in the past to develop a leadership group from within the student body. Efforts had always failed. This year, they blossomed. Several students volunteered to serve on a Leadership Council that tackled some successful projects. Why the difference? The projects were those that mattered to the students. Their input drove the leadership group's agenda. The students on the Leadership Council took data from the Town Meetings and also surveyed students about project priorities, using the information to chose items to tackle first. The initial project created a student eating area. Members of the Council used a quality improvement tool to plan and then establish this new space. They also described their process and results at an FOL Board meeting. The next project, still underway, is recycling aluminum cans to raise money to open an on-site store where students can buy paper, pens, and other school supplies. The evidence is anecdotal but, nevertheless, important: Students exhibited a greater sense of community and program ownership than in past years. While student-centered' has always been important to our partnership, we found a new way to put it into practice.

Our testing program produced data showing that our students are improving their reading, writing, spelling, and math skills. Preliminary results indicate that students gained an average of one grade level in each of these areas after 100 hours of classroom instruction. These data have helped us to identify strong teaching techniques and discard the less effective. The information has been heartening to students and teachers, who have objective evidence that their work is paying off. It has also been helpful in fundraising, since we can show prospective donors that investing in our programs will get results. We have more credibility with these data, and are more competitive.

Reflections

There is an old saying: "It's more important to be lucky than good." We knew we had been lucky, and we thought we had been good. But we wanted to be better. We decided that good intentions, luck, and ad hoc methods had taken us as far as we could go. We could no longer excuse our shortcomings by saying that ours was a fluid' or dynamic' environment, or one that was underfunded, implying that somehow our expectations should be lower as a result. It was time for accountability and documented results through systematic program improvement.

This has been a re-education for all of us. The vocabulary, which comes from the business sector, is new to us. The process can also be discouraging, especially at the beginning when scrutiny revealed some things we would rather not have seen. Indeed, when our Think Tank sat down to identify the "vital few" areas to address rst, we came up with a list of "vital many" and had difficulty prioritizing where to start.

Commitment

This process requires the belief that it will work as well as the commitment of time to think, plan, and involve students, staff, board members, and volunteers. All of these groups have been involved and supportive. The Baldridge Framework is a very logical approach, built on common sense. For those of us geared toward instant results, it has sometimes been frustrating to slog through the tedious process of documenting, in detail, the way we do things (our "as is" state) versus the way we should do things (our desired state). It is time consuming and requires discipline. Just recently, three staff and one board member spent an hour discussing and writing down the process we will use to handle receipts and expenditure. Now we have a simple procedure that everyone involved has agreed upon. It replaces a hit-and-miss method that no one understood. The people involved had to learn process charting and develop this particular chart in addition to their other duties, not in place of them. If the investment of time produces a better process that, in turn, produces better results in this instance, fewer mistakes, less confusion and duplication of effort, more reliable financial data then it will save future time and effort. We do not worry about getting so wrapped up in process charting that we forget the ultimate goal is to achieve results in student performance, because student feedback and student testing helps keep us focused.

Our world is more complex now because we understand how the various parts of our program are interrelated. All our staff and volunteers need to be open to change and to soliciting and receiving feedback. If we have used customer input to design the process, and measured and analyzed both customer satisfaction and results along the way, we should arrive at our goal. If we do not, we have a good idea of what went wrong so that we can improve the next time around. We now understand that orderly is not synonymous with bureaucratic.

We have a long way to go. In the coming year, we will be working hard to bring the ABED and FOL leadership further on board. Although both have been supportive, both groups need a better understanding of the significant organizational implications of this continuous improvement process. We will be implementing more fully our EFF curriculum. We will continue work on our action plans for the "vital few" and document our progress toward improvement.

Deciding to improve the performance of an organization in an orderly, comprehensive, systematic way is analogous to an individual deciding to lose the ten pounds that a stop in front of the mirror revealed. It entails hard work, discipline, incremental results, motivation through small victories and, most importantly, a change in lifestyle. Just as a fad diet does not produce long-term success in weight reduction, fad quality' is not the answer for improving performance over time. An organization must live and breathe quality and incorporate it as a cultural value in everything that it does. We have only begun that process, but, as the saying goes, "a journey of a thousand miles begins with a single step."

 

About the Authors

Kathleen B. Hayward currently serves as President of Literacy, Inc. She is also a licensed attorney with a background in human resource management.

James D. Ford is an adult education teacher and a quality advisor with the Knox County Schools Adult Literacy Program and Friends of Literacy, Inc. He has a background as an Air Force academic instructor and as a literacy volunteer.

Jane K. Cody is Executive Director of Friends of Literacy, Inc, and Literacy Coordination with Knox County Schools Adult Literacy Program. She has more than 15 years experience as an adult education teacher and administrator.

Updated 7/27/07 :: Copyright © 2005 NCSALL