Improving Policy and Practice
ISDDE's Third Goal
Hugh Burkhardt
Shell Centre for Mathematical Education
Abstract
i
The goals for STEM education are largely agreed, nationally and internationally. Work over the last 30 years in the
research and development community, epitomized by ISDDE, has shown how to develop tools and processes for teaching,
assessment and professional development that enable typical teachers to teach much better mathematics and science much
more effectively. Why is this not reflected in most classrooms, and what can be done about it? This paper presents an
analysis of the barriers to progress and an explicit model for tackling them. The lack of synergy between the worlds of
policy, research and development, and practice is described, and explained in terms of the week-by-week pressures that
divert attention from the stated common goal of educational improvement. The proposed way forward, based on the
structures in medical research, tackles the different timescales by proposing three complementary strands of funded
research and development that will offer policy makers well-engineered options from which to select policy initiatives.
The model for change is exemplified here in a paper entitled "Towards Research-based Education: integrating policy,
research and practice", written for policy makers and their advisors in the U.K. The intention is that others might
adapt this model paper for their own contexts.
Introduction
ii
For at least a decade I have believed the growing evidence that, as an international community:
We know how to enable typical teachers to teach better mathematics much more effectively
but that
we do not know how to get real school systems to change so as to achieve this.
How we might do better is the focus of this paper.
ISDDE brings together much of the best in design and development from around the world – work that has the potential
to improve STEM education. This was implicitly recognized with the award by ICMI in 2016 of its first global prize
for "excellence in the practice of mathematics education" to a team (Burkhardt
and Swan, 2017; ICMI, 2016) that
reflects what ISDDE stands for: a design approach that, using engineering research methods, works to improve
standards of teaching and learning. This recognition was not inevitable. ICMI could have chosen, for example,
outstanding practitioners in supporting teachers - like Emma Castelnuovo, after whom the prize is named. This
suggests real progress with the first two goals of the Society:
- improving the design and development process
- building a design community.
But we have made much less progress with the third goal
-
increasing our impact on educational practice,
and on the policy frameworks within which practitioners operate. What might we do to change that?
Here I set out an analysis of the problem and propose an explicit model to tackle it. The model takes a novel
perspective in treating politicians and their advisers as a part of the system we seek to improve, not as a set of
given external forces and constraints. Indeed, if we cannot move these in the right direction, the impact on
practice is likely to be limited in scale - as so often it has been in the past.
Some history of the model
iii
The argument here has had a long gestation with many influences. It began to crystallize with my ISDDE Prize talk at
the 2014 Cambridge Conference, entitled
Making education more like medicine: strategic design for research-based
policy.
The panel discussion that followed, among people involved with policy formation, made very clear the
complexities of the system. From them I saw I had a lot to learn about that world and the processes from which
policy emerges. Since then many people with high-level experience of how government works have contributed to my
education - still far from complete. This process culminated in the Shell Centre Jubilee Conference in 2017, on the
related theme Impact-focused research - how can it become more effective? (MARS/Shell Centre, 2017). After that it
seemed time to develop the analysis as far as I now can, and to share the results.
I have wondered how best to present these ideas to the design community. Should I write a detailed discussion of the
issues and possible approaches? However, I have long found that examples communicate more vividly and with less
ambiguity than descriptions. It is easier to modify an explicit model than to create one from generalities. So I
decided to develop a model paper aimed at policy makers and their advisers in the U.K., presenting it to
Educational Designer and its readers as a case study, adding only this context-setting introduction.
An outline of the argument
iv
The model paper is entitled Towards research-based education - a title chosen to have wide appeal, in particular to include U.K. policy makers.
It can be downloaded from:
educationaldesigner.org/ed/volume3/issue12/article46/pdf/towards_rbe.pdf
After an executive summary and an outline of the argument (U.K. policy makers like 1 page + 5 page + 25 page papers) the sections are:
- Is there a problem?
- The policy makers' world
- The education research world
- The world of educational practice
- How could 'the system' work better?
- The initial concerns addressed
Appendix: What does good engineering look like?
Sections 2 to 4 reflect a widely-shared consensus - that the three worlds of policy, education research and
educational practice, do not communicate well enough to work effectively together towards improving teaching and
learning in classrooms. Though rhetorically they have the same goal, pressures in their working lives give each
group other, more compelling priorities. More broadly, the problem is exacerbated by mismatches of timescale - the
decade timescale of real educational improvement against the week-by-week pressures on the professional lives of
politicians, researchers and teachers.
Aspects of particular concern that emerge include:
- Poor communication between politicians and policy makers and the education communities.
- Trying to 'fix the problem' - a political tendency that fails to take into account system complexity, or to
recognize that real improvement, involving changes of well-grooved professional practice, is inevitably gradual
and complex.
- Technical naïveté - the tendency of politicians and policy makers to prescribe aspects of teaching and
assessment at a level of technical detail that they would not dream of trying in, say, medicine or engineering,
thus discounting the expertise of the education professions.
- Pressures for uniformity - the limited opportunities for and shortage of support for pilot projects that can,
after evaluation, grow into and improve the mainstream.
- Imbalance in education research between the dominant analytical-diagnostic research traditions and
treatment-focused research and development with an engineering approach.
- No generally accepted body of results - the failure of the education research community to develop, on the one
hand, a solid body of agreed research results and, on the other, detailed evidence on the effects of specific
'treatments'.
- As a consequence, the lack of authoritative structures that integrate evidence from research and practice in a
form that policy-makers respect and can use.
In the analysis, and the ways forward that I propose in Section 5 of the model paper, I have been influenced by the way research supports policy and practice in medicine, and how that deals with similarly different timescales.
Using the model paper
v
The aim of the model paper is to stimulate a dialogue that goes beyond generalities towards testable models, in the
engineering sense, of how we might overcome the system barriers to progress.
What has been the response to this initiative? All one can say so far is that it has not been the silence that I
half-expected. Government has shown some interest. Of course, nothing has actually happened yet, but the dialogue
has started and initial plans are being made. There have been positive responses, particularly from the scientific
community, which seems to recognize the problem and has an active interest in finding ways to do better. Shortly
after the U.K. paper was released in August 2018, the Royal Society and the British Academy launched a report,
Harnessing Education Research, (Royal Society and British Academy, 2018) which, though the emphasis is different
and the proposals less specific, is broadly in harmony with my paper.
How far are these ideas and the model paper that presents them generalizable to education systems in other
countries? While the challenge of turning potential improvement into classroom reality 'at scale' is similar in
many countries, the factors involved in doing so vary substantially. The policy making world in the U.S.A., for
example, has an apparent diversity that sometimes seems to border on chaos; very different from the highly
centralized systems in the U.K. Yet there is a similar mismatch between the goals and the reality in most
mathematics classrooms. (On the positive side, there are so many separate school systems with which one might
develop a model!) I plan next to work with American friends to see if we can develop a version that makes sense for
U.S. policy makers.
You are here also invited to join in the discussion and indeed adapt the model paper to advance educational goals in
your own setting. Beyond adaptation for the different educational structures, histories and priorities in other
countries, quite different approaches will, of course, be welcome additions to the discussion. In particular,
Educational Designer has always invited considered responses to articles. If someone from another country likes to
take up the challenge of developing a 'local version', that's great - and I am happy to cooperate.
Everybody is therefore welcome to copy, use and adapt the model paper - please retain the reference to this article by way of attribution.
Acknowledgements
vi
I owe enormous thanks to many policy 'insiders', particularly Roger McClure, David Laws, Alaster Smith, Neil
Carmichael, Stefano Pozzi and Charles Clarke who all helped my understanding of policy making. I have learned from
discussions of the change process over many years with Alan Schoenfeld, Phil Daro, Mark St John, Kaye Stacey, Jan
de Lange, Celia Hoyles, Andy Noyes, Geoff Wake, Lynne McClure and other colleagues - in the Shell Centre and
beyond. Thanks to them all; they are not, of course, responsible for this paper.
References
vii
Burkhardt, H., & Swan, M. (2017). Design and development for large-scale improvement. Emma Castelnuovo Award lecture
in G. Kaiser (Ed.) Proceedings of the 13th International Congress on Mathematical Education, pp 177-200. Cham:
Springer International Publishing
About the Author
viii
Hugh Burkhardt has, since 1976, led the Shell Centre team in a series of integrated curriculum and assessment projects
to develop tools that help teachers to align their practice with international learning goals in mathematics. He takes
an "engineering" view of educational research and development: that it is about using prior research, imaginative
design and systematic development to make high-quality tools that help a complex system work better. A theoretical
physicist, he has always seen using mathematics to tackle everyday life problems and decisions as central to a
high-quality education. He led the foundation of ISDDE, and of Educational Designer. Based on this work over 35 years,
he and his colleague, lead designer Malcolm Swan, were chosen by the International Commission for Mathematical
Instruction to receive the first Emma Castelnuovo Award for Excellence in the Practice of Mathematics Education.
In 2013, Hugh was awarded the
ISDDE prize for Excellence in Educational Design based on his work at the Shell Centre for Mathematical Education.