|Accomplishing Large Scale Reform: A Tri-Level Proposition|
Ontario Institute for Studies in Education University of Toronto
Prepared for the Journal of Educational Change, November 2001
A critical new theme of the 1990's was how to achieve large scale reform. In the current decade sustainability has been added as a major concern. These twin concepts represent a radical shift from understanding individual school innovation toward establishing system change that generates and supports continuous improvement on a large scale.
In this paper we use literacy and to a certain extent numeracy initiatives as examples of attempts at large scale sustainable reform. We first describe the sources we use from our own and others work — a lively body of multi-year attempts at large scale reform. Second, we offer a tri-level model — school/district/state, along with evidence to demonstrate what is necessary at each of these three levels in the pursuit of system-wide reform. Third, we identify an agenda of unfinished business in order to take us to the next level of sustainable reform.
We should also provide an advance organizer for what we mean by large scale, sustainable reform. We exclude for example, large scale external reform models such as Success for All, even though they are underway in thousands of schools. There are two reasons for this exclusion. First, these school-wide models, while comprehensive, are not occurring in school systems, i.e., they are situated in thousands of ad hoc schools. Second, they represent externally adopted models as such, and are not likely to produce deep change in the culture of learning. At best, the models get implemented, but do not produce the kind of deep cultural change required for continuous improvement. We acknowledge that they represent legitimate large scale reform (for an excellent study of this set of models see Datnow et al, 2002). It is just that we do not believe that they represent the future because they can never produce deep organization and system change.
We do not attempt a systematic review of research (see Fullan 2001a, 2001b). Rather, we describe some case examples of large scale multi-year case studies, many of which we are currently involved in. This is the data base for this article. In particular, we include the reform work in District 2, New York City, and in San Diego as well as our own training, research and critical friend roles in three districts in Canada (Edmonton Catholic Schools in Alberta, the Toronto District and the York Region in Ontario). Finally, we report on our evaluation of the National Literacy and Numeracy Strategy in England.
First, we build on the excellent work of District 2 in New York City, coupled with San Diego City School District. What makes these two cases interesting is that they are sequential attempts at achieving ever more complex reform using essentially the same set of strategies.
District 2 in New York has fewer than 50 schools. In 1988 it ranked tenth in reading and fourth in mathematics out of thirty-two sub-districts. Using a systematic reform strategy based on seven themes, eight years later, by 1996, it ranked second in both reading and mathematics. Elmore and Burney (1999) identify the seven organizing themes or principles of the strategy: (1) it's about instruction and only instruction; (2) instructional improvement is a long, multistage process involving awareness, planning, implementation, and reflection; (3) shared expertise is the driver of instructional change; (4) the focus is on system-wide improvement; (5) good ideas come from talented people working together; (6) set clear expectations, then decentralize; (7) collegiality, caring, and respect are paramount (p. 272). These themes were instituted through a subset of strategies which include: intervisitation (teams of principals visiting schools to examine implementation of initiatives), monthly principal support groups, peer coaching, study groups, institutes, and the like (see Fink and Resnik, 2000). This work involves ‘learning in context' — built-in methods for groups to learn together focussing on the actual work of the district. It is moreover systemic - all schools, all leaders, all teachers are involved together.
San Diego represents an interesting case because the leadership in District 2 became involved in designing an intensive reform effort beginning in 1996 and involving all of the district's 187 schools. The focus again was literacy and numeracy. In a sense the proposition was can you do in San Diego with 187 schools what you did in District 2 with 48 schools in half the time by using the strategies more intensely? The short answer is yes (but see our qualifications about sustainability in the concluding section of this paper). In the pre-strategy years (1993-1996) scores in reading and mathematics in San Diego were flatlined - neither increasing nor declining. The new strategies were put in place commencing in 1996-1997, and after a year's lag the results have steadily increased by some 10-30% (depending on the subgroup) from 1997-2000.
Our own reform efforts involve a series of large scale, multi-year projects in which we are serving as trainer and/or ‘critical friend' evaluators/consultants. We cite four in particular.
In the Edmonton Catholic School District in Alberta, Canada, we are engaged in the third year of a multi-year training of school teams from all 84 schools in four cohorts of 21 schools. Each team consists of the principal and 4-6 teacher leaders. The initiative is called Assessment for Learning. Each school uses the knowledge base that we and others have developed to guide their efforts to improve student learning and achievement in targeted areas. The knowledge base includes: understanding the change process, building professional learning communities at the school level, assessment literacy, knowledge building and sharing, the role of the district in fostering school reform across all schools dealing with resistance, and going deeper. In effect, the project has focused on ‘reculturing' the district. We are now conducting a series of case studies (not yet available) to derive lessons and conclusions.
In York Region District School Board (118 elementary and 23 secondary schools), to the north of Toronto we are not doing the training, but serving as a critical friend consultant focusing on both the school and the district level.. First, we have completed six case studies of schools involved in literacy initiatives. Second, we are advising on how the district as a whole can develop a strategy for system-wide change (Mascall et al, 2001).
In the Toronto District School Board, we are involved in the second year of training school leaders from 93 schools engaged in an Early Years Literacy Project. Each school team consists of the principal and a literacy coordinator (.50 position). The content of the training is similar to the Edmonton initiative except that it is all channeled towards improving early literacy. We have just completed seven case studies of schools, which are reported in the next section (Edge et al, 2001). Still to be considered is how to go district-wide in a system that has 451 elementary schools and 102 secondary schools.
We are in the final year of a four year evaluation of the National Literacy and Numeracy Strategy in England (Earl et al, 2001). In 1997 the newly elected Labour government selected literacy and numeracy as priorities for the 19,000 primary schools in England. They established base-line achievement figures (in 1996, 57% of all 11 year olds in the country were achieving at the proficient level in literacy; the mathematics figure was 54%). They set targets for 2002 of 80% for literacy and 75% for numeracy. The strategy to get them there was essentially drawn from the knowledge base we are discussing in this article, combining accountability and capacity-building (see Barber, 2000, and Fullan, 2001a, Ch. 13). We received the contract to monitor the implementation of the strategy, and feed back our findings on an ongoing basis. As of 2001, literacy achievement has risen to 75% (on the way to 80%), and 71% for mathematics (on the way to 75%). The complex issues in this national case are discussed in the next section.
What are we learning about large scale, sustainable reform?
Our argument in a nutshell is that to get large scale reform, you need to establish and coordinate ongoing accountability and capacity-building efforts at three levels - the schools, the district, and the state. We illustrate our findings at each of the levels. We conclude that large scale reform is being accomplished with significant, but not necessarily deep results. Further, the conditions for sustainability simply are not evident.
The School Level In our view the best depiction of what is needed at the school level derives from the work of Newmann and his colleagues (2000). The model they have developed is a compelling starting point (see Figure 1).
Source: Newmann, King & Youngs (2000)
Newmann et al found that school capacity was critical, which they defined as consisting of five dimensions:
Basically Newman et al claim, with backing from case studies, that professional development often focuses on knowledge, skills, and dispositions of teachers as individual staff members. This is the first component of school capacity. Obviously this is important and can make a difference in individual classrooms, but in isolation it is not sufficient (never send a changed individual into an unchanged culture).
In addition, there must be organization development because social or relationship resources are key to school improvement. Thus, schools must combine individual development with the development of school-wide professional communities, the second element of capacity.
However, individual development combined with professional communities is still not sufficient, unless channeled in a way that combats the fragmentation of multiple innovations by working on program coherence, "the extent to which the school's programs for student and staff learning are coordinated, focused on clear learning goals, and sustained over a period of time" (Newmann et al, 2000, p. 5). This third element, program coherence, is organizational integration.
Fourth, instructional improvement requires additional resources (materials, equipment, space, time, and access to expertise).
Fifth, school capacity is seriously undermined if it does not have quality leadership. Put differently, the role of the principal is to cause the previous four factors to get better and better. Elmore (2000) agrees:
[T]he job of administrative leaders is primarily about enhancing the skills and knowledge of people in the organization, creating a common culture of expectations around the use of those skills and knowledge, holding the various pieces of the organization together in a productive relationship with each other, and holding individuals accountable for their contributions to the collective result. (p. 15)
We will see that this model has been verified in our own case studies. Before commenting on these findings, however, we need to comment on what is missing or undeveloped in the model. Three key things. First, the parents and the community are omitted. We know that reform will not be successful unless the school can develop a strong, mutually influential relationship with the community (Fullan, 2001a, Ch. 12). Second, in our own work, 'assessment literacy' is a central strategy (it is implied in Newmann's model under instructional quality). Assessment literacy as a strategy involves developing the capacity of teachers and principals collectively to:
Third, the external infrastructure at the district and state level is largely missing (it is partially included in the bottom box 'Policies and Programs'). It is this infrastructure which constitutes the second and third levels of our tri-level model as we discuss below.
School Level Lessons
Focusing on the school level for the moment, in the six case studies in the York Region District School Board, consistent with Newmann et al, we found that all five aspects of school capacity were associated with success (teacher skills, professional learning community, program coherence, resources, and principal leadership). However, we were able to identify additional nuances. First, as in all our studies it is not just principal leadership that counts but the combination of instructionally focused principal leadership with one or more other change agents inside the school. In York Region this meant the principal, the Mentor Teacher (as the literacy coordinator was called), Reading Recovery teachers; and in some cases the vice-principal.
Second, in four of the six schools highly collaborative cultures (professional learning communities) were evident. We emphasize that this is not individual professional development, but shared development in which teachers meet frequently, discuss challenges particular children are having, and support (and pressure) each other. In three of the four collaborative schools there also was strong evidence of 'assessment literacy' as staff a analyzed and interpreted student data and used this to alter their practice. It is crucial to understand that this is learning in context, i.e., what is learned is specific to the school situation, and it is done collectively, it is shared.
Third, program coherence or focus was critical but difficult to maintain. Schools are under constant pressure to juggle multiple initiatives. Even the literacy strategy had several different components, which needed more integration.
There were also difficulties. These included:
Many of these findings are corroborated in the case studies of the seven schools in the Toronto District, and so we won't repeat them in detail. Once again we found that the combined leadership of the principal and the literacy coordinator was crucial. We also found a number of issues that had to be sorted out with respect to the role of literacy coordinator - the clarity of the role; relationships with other teachers in terms of trust, expertise, and age; relationship with the principal. Similarly, resources, maintaining focus, reconciling different assessment techniques, coping with turnover, and maintaining momentum were all issues of concern.
All and all in both the York and Toronto projects, schools have made considerable progress. Those in year two or three of the initiatives were especially effective, reflecting what is normal in many large scale reform efforts. In year one people experience the difficulties of getting started, and some misgivings about the top-down nature of the strategies (remember we are talking about large scale reform); in year two (if the strategy is sound) people talk about initial success; by year three people can see that their own skills, especially the collective skills of teachers and principals together, have developed. They see results of their efforts, can pinpoint problems in student learning, and have greater confidence about how to address the problems. (This is also the case in the Edmonton initiative).
Two big problems remain. In both districts only about a quarter of the elementary schools in the district were engaged in the projects. Going to scale remains an issue. Second, even in those schools in which success was being experienced three years into the initiative, nay, especially in those schools in which success was evident, the big worry was 'sustainability'. People were right to worry, because neither achieving nor sustaining large scale reform is possible unless the district and state get their acts together.
We have written elsewhere about the role of the district (Fullan, 2001a, Ch. 10). And, certainly the principles and strategies used in District 2 and in San Diego provide some clarity about what districts need to do. We highlight in this section some of the key requirements of effective districts and difficulties they have in staying the course.
We start by observing that if school capacity is critical, the main objective of the district should be to generate and maintain greater capacity in all or in the vast majority of schools in the district. Here are some of the ways in which districts can address this issue.
First, start with literacy (and we could say numeracy). It is essential that these foundational skills be established as building blocks for other subjects and developments. This involves establishing an accountability/capacity-building initiative across many schools as we have seen in all the districts. The key point is that districts establish instruction as the priority. By focusing on instruction/curriculum, districts embed their pressure and support solely in the service of improved teaching and learning.
Second, a critical part of the strategy involves directly investing in leadership roles at the school level (e.g., principal and literacy coordinator) as well as appreciating that such an investment also pays off down the line. For example, think of the 93 literacy coordinators in the Toronto District receiving great training and experience as 'change agents'. In turn these individuals are likely to be the leading candidates when positions become available at the vice-principal and principal levels (assuming that the district is seeking principals as instructional coordinators). We only have anecdotal data but we would hypothesize that experienced literacy coordinators are becoming the leading candidates for principalships. If they get promoted in numbers, they will in turn develop the capacity of teacher leaders in their schools at a much greater rate than their principals did with them. Soon a critical mass with a steady pipeline of leadership development will be in place.
Third, recognize the community-building nature of learning in context. Such learning is specific and it fosters sharedness. It takes place within school districts and schools, but deliberate strategies must be established in the overall district design so that learning across schools is featured. This begins to foster commitment to other schools and to district success as a whole.
Fourth, focus on assessment literacy, benchmarks of achievement, and a new indicator that we are suggesting as the true measure of progress - closing the gap between high and low performers (school to school, group to group). Closing the gap is the greatest contribution schools can make to societal development. It involves raising and leveling the differences as all schools move forward with low performers moving at a greater rate.
Fifth, intervene in schools which are persistently failing in order to help them to move forward. The goal is to take action in order to move schools to a level of capacity where they can go forward more on their own (always in the context of district stimulation).
Sixth, conduct an inventory of district initiatives with a view to achieving greater coherence or connectedness. Sometimes this means dropping certain activities, other times it involves consolidation or integration. Working on coherence making is the greatest need for complex systems (Fullan, 2001b). San Diego is a good example. Prior to the 1996-97 focus, San Diego was a highly innovative district. The problem was that it was too innovative. It had multiple disconnected initiatives that came and went at irregular intervals. It needed to consolidate and focus, and that was what the new leadership did.
District Level Lessons
In York Region and in Toronto we see the initial success of the literacy projects now reaching a crossroads. Will these successful endeavors, which are currently not integrated into district wide systems move to the next level of incorporation or will they become another example of 'this too shall pass'. York Region, for example, has much going for it. The literacy initiative is successful in terms of raising literacy achievement. It has a number of other quality initiatives that feed forward in the same direction of capacity-building. Our recommendations to York Region were the following:
The Toronto District is much larger and presents a more complex set of problems. Among other matters the District needs to continue and expand the work of the 93 schools to other schools in the district. The investment in training and support of the school teams (principal and coordinator) has been very effective. Another issue is how the early literacy program can be integrated and supported by area superintendents across the district. At the present it is lead by one area superintendent coordinating the work across other area superintendents.
The biggest problem in the Toronto District is working through the aftermath of amalgamation in which seven districts were incorporated into one (unlike York Region which was unaffected by amalgamation). The amalgamation has accelerated staff turnover. The District Director (Superintendent) has just resigned to take another position.
Our point is that it is impossible to develop school capacity across the vast majority of schools, i.e., it is impossible to accomplish large scale reform, if the district does not improve its own capacity. Infrastructure counts. It can lead the way or it can actually undercut efforts of individual schools on the move, while neglecting other schools that are persistently failing.
So far we have said that the first two levels, schools and district, must work in a mutually beneficial direction, and we have provided some examples of districts moving down that path. Now we say, districts cannot play this role if the state is not doing the right things - the third part of our tri-level model.
Just as schools will not develop capacity if districts are not helping (or if a few do, it won't be sustained), districts will not progress if the state policy context is not working to foster district and school development. This means that the state must work to establish a sophisticated blend of pressure and support (or accountability and capacity-building). In this section we illustrate what this looks like in terms of what we will call the specific infrastructure (i.e., specific to literacy and numeracy), and the generic infrastructure (i.e., policies related to the overall quality of the teaching profession).
We take the National Literacy and Numeracy Strategy in England as the case in point. When the Labour government came to power in 1997, they established literacy and numeracy as top priorities. As we saw earlier the government established baseline measure (the percentage of 11 year olds performing proficiently) and new targets to be met over a five year period. They drew on the knowledge base about change (again pressure and support), and crafted a comprehensive strategy. Michael Barber, the head of the government initiative describes the main elements of the implementation strategy:
(Barber, 2000, pp. 8-9)
Note, the blend of pressure and support, and problem-solving mechanisms. Most governments invest in accountability (pressure) but not in support (capacity-building). From the start, the English government made substantial new financial investments. It is also revealing that as achievement targets began to rise, additional monies were garnered. In other words, each degree of success was used as a lever to obtain more resources from Treasury. It is also noteworthy that demonstrable success was obtained within one electoral four year term, and was one of the factors instrumental in the landslide 2001 reelection, which brought even more resources (and continued pressure). We will qualify our interpretation of success in the final section of the paper, as there are still some fundamental problems. As a first phase, however, the English case represents an impressive accomplishment.
The generic infrastructure is another matter. Here the question is are the state policies (concerning compensation, standards of practice), and working conditions for teachers and administrators such that the quality of the teaching profession is enhanced? Measures of enhancement include good people coming into teaching (and staying); morale; and continued development of the quality and performance of schools. In a sense, the role of the generic infrastructure is to contribute to accountability and capacity developments on a large scale with respect to the previous two levels (schools and districts). The empirical question is does the generic infrastructure enhance quality performance or fail to enhance it? We have to say that in most jurisdictions including England, the generic infrastructure has so far failed to make a difference as the system continues to weaken (or at the very least not move from a weakened to a stronger state).
The key policy strategies with respect to the generic infrastructure include (among other things): the quality of initial teacher preparation; progress; induction; continuous professional development tied to standards of practice; compensation for teachers; the recruitment, continuous development and retention of leaders (as school principals); and the alteration of the working conditions of teachers toward creating professional learning communities that mobilize and engage teachers, parents, business and community leaders in the services of student learning.
Using England as the example, the generic infrastructure has not yet improved as indicated by an increase in the attraction and retention of more teachers, teacher morale, more effective school and district leadership, and so on. If anything the specific infrastructure has weakened. Consequently large scale, sustainable reform is not possible. The next steps, then, are crucial and they are not straightforward. We turn to some of these key issues in the final section of the paper.
We have made the case that new capacities have to be built at all three levels, and we have provided evidence of good work happening at each level. We do not, however, have evidence of the three levels working in concert. And indeed our overall recommendation is that policy makers need to turn their attention to developing capacities and interactions across the three levels if they are seeking large scale, sustainable reform.
There are four main aspects of the unfinished agenda, and a final caution we would offer. First, concerning literacy and numeracy, a set of policies on accountability and capacity-building must be established that take into account all three levels and their interrelationships. We have outlined in each section what that would entail.
Second, also concerning literacy and numeracy, it is important to worry about the limitations of a tightly orchestrated tri-level strategy. As successful as the first 5 years of the English strategy has been, there are fundamental doubts about whether that strategy is appropriate for going to the next level of reform. Among other things the English strategy has supplied lesson plans and resources on the web. We have said earlier that this has helped weaker teachers. The question, however, is whether all or most teachers start to use "provided" materials because that is easier and because they wish to cover themselves. Such mechanical following of central directives is more likely as the government sets new targets for 2004. Following the election in May 2001, and following a year of non-movement in assessment scores (literacy was at 75% in 2000 and stayed at that level for 2001; numeracy went from 72% to 71% in the same period). The danger is that even more intensified, prescriptive high pressure strategies will be used, and what is worse, teachers will be vulnerable to following directions. The overall strategy has given teachers a lot of new information and good ideas, but given that, the next phase should be based on giving teachers (not as individuals, but as professional learning communities) time to reflect on, apply and consolidate what they have learned. It is time for schools (principals and teachers) to make the strategies their own, not to become even more government-directed.
This brings us to our third point which concerns the generic infrastructure and the quality, morale and internal commitment of the teaching profession qua profession. In acknowledging England's first phase success in improving literacy and numeracy, Baker (2001) makes the case: In countries where accountability measures have undermined teachers' autonomy, there is now a recruitment crisis...
So this is Britain's cautionary tale: Policymakers must involve teachers in the reform process, and accountability must be balanced by professional autonomy. In the past, teachers in England had high autonomy and low accountability. The past decade has produced a tilt to an opposite imbalance: low autonomy and high accountability.
The result has been a demoralized teaching profession. England has now started to emerge from the rapids of school reform. There are sound structures in place for future progress; but just as the government hoped it could build on these new foundations, it was hit by the crisis of teacher recruitment.
What both the United States and the United Kingdom need is a balance: both high accountability and high autonomy for teachers. Not one or the other, but both.
The warning is there. Somewhere along the road of England's school reforms, the policymakers took their eye off the ball. It is as if the football coach had worked out the most careful and detailed theoretical plays only to look up, on the day of the game, to find his [best] players had lost interest and gone home with the ball (pg. 36).
A word about professional autonomy. Our version is one steeped in professional learning communities in which lateral accountability (as teachers focus collectively on student learning and what it will take to get there) among teachers is enormously powerful. No loss in accountability there!
The fourth point concerns broadening the curriculum beyond literacy and numeracy. There is a great deal of evidence that certain sets of life performance dispositions and skills are required for the knowledge economy of the 21st century. - problem-solving in novel situations, teamwork, emotional intelligence, good citizenship, commitment to life-long learning, and the like. You can't get them through prescriptive methods. Policymakers must begin to focus on these developments with the same intensity as they did for literacy and numeracy. Teacher ownership will be even more crucial in these domains.
The caution. Change in complex society will never be linear. So don't expect a tri-level coherent system that settles down once and for all (see the Change Forces trilogy - Fullan, 1993, 1997, forthcoming). But successive approximations are possible. Whatever level in the system you are at, work on the tri-level agenda. To be content with your own bailiwick is to make large scale, sustainable reform impossible. And indeed, to confine local reform to episodic spasms.
Baker, M. (2001). Accountability vs. autonomy. Education. Washington, DC: October 31.
Barber, M. (2001). High expectations and standards. Unpublished paper. London: Department for Education and Further Employment.
Datnow, A., Hubbard, L. & Mehan, H. (2001). Extending educational reform. London: Routledge Falmer Press.
Earl, L., Levin, B., Leithwood, K., Fullan, M., Watson, N., with Torrance, N., Jantzi, D. & Mascall, B. (2001). Watching and learning 2. Toronto: Ontario Institute for Studies in Education, University of Toronto.
Edge, K., Rolheiser, C. & Fullan, M. (2001). Case studies of literacy-driven educational change. Toronto: Unpublished report to the Ontario Ministry of Education.
Fink, E. & Resnik, L.B. (2001). 'Developing principals as instructional leaders', Phi Delta Kappan, pp. 598-606.
Fullan, M. (2001a). The new meaning of educational change (third edition). New York: Teachers College Press; Toronto: Irwin Publishing Ltd.
Fullan, M. (2001b). Leading in a culture of change. San Francisco: Jossey-Bass.
Fullan, M. (1993). Change forces: Probing the depths of educational reform. London: Falmer Press.
Fullan, M. (1999). Change forces: The sequel. Bristol, PA: Falmer Press.
Fullan, M. (forthcoming). Change forces with a vengeance. London: Falmer Press.
Mascall, B., Fullan, M. & Rolheiser, C. (2001). The challenges of coherence and capacity. Toronto: Unpublished report for the York Region District School Board.
Newmann, F., King, B., & Youngs, P. (2000). Professional development that addressess school capacity. Paper presented at the annual meeting of the American Educational Research Association.