Why results can't be used to measure teachers and students

Yes, it’s true, using the exam results that students achieve as a measure of the abilities or qualities of students and teachers, is impossible and a complete contradiction. It amazes me that anybody still thinks you can.

As Graham Nutthall points out in his intriguing summary of his life’s worth of studies called ‘The Hidden Lives of Learners’ (thanks to @MrDMJWalsh for the tip-off), you can’t do it.

Let me explain why:

Measuring teachers through results.

If good results mean the teachers are good, then universities, parents or anyone else can’t use exam results as a measure of the students’ abilities, as it wasn’t their abilities that led to the good results. It was the result of the good teachers they were lucky enough to have.

Measuring students through results.

If good exam results are as a result of the students being good, then they can’t be used to measure teachers by, as it is a reflection of the students they were lucky enough to be teaching. Not their own teaching ability that led to the favourable results.

www.youtube.com

Of course, I can’t imagine that anybody would argue that results are solely down to just the teachers, or just the students. But the performance of one group affects the results the other is measured on. This performance is a factor that is largely, or completely, out of their control. As I said, it amazes me. It is madness.

Even so, every education system I’ve ever comes into contact with does it. Remember, most assessments are incredibly narrow. Most measure academic ability or intelligences without exploring the other skills and attributes students may have been developing.

But if the focus of assessments is narrow, and the results of the assessments are high stakes enough, for the teacher and student there are huge temptations.

Stories of students trying to cheat have rarely been in short supply. But as Levitt and Dubner illustrate in ‘Freakonomics’, the pressure can get to teachers too. They managed to prove beyond doubt that some teachers had been cheating by changing students’ answers on test papers. They could do this by applying economic data analysis tools to discover unusual answering patterns on test papers. But, there are far more subtle ways that high stakes testing changes learning in schools. And they may even be more damaging.

‘Teaching to the test’ becomes irresistible as more and more pressure is piled on to get results. This must surely be to the detriment of the wider learning of the students?

Weekend exam revision sessions and extra tutoring in the evenings crams students’ heads with stuff to be vomited up through their pen onto test papers, before being quickly forgotten. And what life experiences are the students missing out on while doing all this extra exam preparation? What is the measure of the impact of not doing their sports, socialising, sleeping enough or relaxing? Nobody seems to care.

pixabay.com

So it is time to accept that you can’t measure students and teachers on results. I think you can’t measure people’s abilities on something they don’t have control over, it’s nonesense. And placing huge pressure by making them high stakes is detrimental to students and teachers. Time to find something different.

Any ideas?

Preserving the Process

Ever heard of a WAAG? I have to admit I hadn’t, until our new deputy started to send around an email called ‘Week At A Glance’ with all the notices and events on it. It’s pretty handy. But the reason I mention it is because recently she has been adding a little image at the top, a different one each week. Often to do with something happening that week, like a picture of a visiting author and a book she has written. However, the last two have been reminding folk of the ‘fixed V growth’ mindset discussion that is en vogue just now.

It has been interesting watching the growth of this thinking over the last few years. I find it fascinating how it has been taken by various media outlets, misrepresented and misinterpreted so much so, that Carol Dweck herself has had to update what exactly she meant. She has also added a few more points. We have the ‘false growth mindset’ as an idea that people know what they should say they are, and declare they have the Growth Mindset, but in fact their actions paint a different picture.

So where am I going with this? I don’t want to discuss the Growth Mindset itself, that has been done plenty of times by others already. In general, I like the concept, it makes sense to me, and I use the terminology with my students and myself. But, how can technology aid, clarify and articulate the process of developing a Growth Mindset?

There is nothing better than having to learn something new, to test out your approach to learning. And technology delivers new learning in abundance. For me, it is particularly interesting observing the teachers as they learn. The whole range of approaches surface, those who give up easily, those who know this is going to take lots of effort, and calculate that the outcome is not worth their effort, and those that just get on with it. Some just want me (or a colleague) to show them the steps, so they can get it done. But in reality, we know that is not an effective way to learn. The students, in my experience, are far more resilient learners when it comes to using new technology.

developer.apple.com

But I believe the true strength of technology in regards to this lies in the ‘Mutability’ element of the ‘SAMMS’ model of transformation. The ability to revisit, edit, change and adapt work. But even better, the ability to document this process of creation. My students have their own personal blog, as well as the class one. Here they showcase their final pieces of work, and reflect on what they have learnt. But I’m starting to think we are missing a trick. Why do only their final pieces of work get to be displayed? Why not all the previous incarnations, so we can see how the creation has developed?

So my next task, is to work out a way of concisely showing the process of learning and creation that has gone into each final piece. This way, they can demonstrate the mistakes that they made, the problems that they solved and the different approaches they tried, as well as any help they sought and received.

Any ideas?

The Future Of Ambiguity

Apple state that when they are looking for new staff there is one quality they seek more than any other; the ability to cope with ambiguity. Thinking about the skills students need in their future careers, it seems reasonable to expect that certainty will be in even shorter supply than it is now.

When folklore and superstition ruled the planet, what the village elder/witch doctor/ soothsayer declared, was the truth. And truth was certain. During the industrial revolution, you followed the instructions given to you by the foreman and income, shelter and food were relatively certain. Now, is anything certain? Perhaps it never really has been, but there was a definite illusion that it was for many ordinary folk. This illusion has been shattered now, ignorance is no longer a shield to being aware that things could easily change, and probably will.

Information Edmition

So, when it is a natural instinct for many to seek out certainty, how can we prepare students for living in uncertain times? The ‘pre-school, school, college, university then job’ treadmill is no longer a certainty. A good education does not guarantee a job for life anymore, or even a job.

I would argue that teachers have been dealing with ambiguity for many years. Measured upon delivery of success using the narrow indicators of exam results, or grades, it has become a working truth for teachers that these are not the most important indicators. In fact, it is clear to anyone who thinks about it that, exam scores are a shallow indicator of an effective teacher. They have to pick out the best route from the ambiguity of options through the content, skills and understanding that they think their students are most in need of, for their future careers. Balancing the need to meet targets, with the desire to do what is best by their students.

Advice on how teachers should be going about their role is everywhere they face. In the media, in books, from parents, from politicians, from school leadership, at courses, on INSET, from inspection agencies, even from colleagues and now all over social media. A lot of it is not that bad, can be effective and is usually well meant. But it is also often contradictory. If nothing else, effective teachers are masters and ensuring they do the things they are required to, which are beyond their control, then selecting the tools that will serve their students best from the plethora that they are faced with. Choosing what to ignore, what to disregard, must be an essential skill in dealing with ambiguity.

What Apple needs, is more teachers!

Create & Construct by lorraine santana, on Flickr

In fact, the real disservice for students comes from only presenting them with certainties in their studies. For example, students are often told; fulfil this certain set of criteria, and you will be certain of this grade. Of course students do need to know what they need to do, and what a good piece of work looks like. But they must also be set tasks where the outcome is uncertain. Where there is no one correct way of doing things. This is a more true reflection of the fluid workplace that so many will be working in. It is their job to come up with an end result without knowing what it has to look like..

For teachers, supporting this must be a priority.

Innovation / Consistency Connundrum

“Have you seen what teacher Y does with (insert name of APP/ technique here)?

It's amazing, they have started using it in a way I would never have thought possible. And teacher X has thrown out doing (insert name of mandatory school-wide protocol) because they thought they would try a different way.” said not very many people ever, probably.

missionfitonline.co.uk

It is a constant balancing act between innovation and consistency that I see regularly in schools and teams within those schools. The community and administration (not necessarily always ‘The Administration’ though) require predictability and consistency. People need to know where they stand, where to get information and how communication will work. As part of a community it is respectful and necessary to be predictable in the way you go about your work. It enables others to make decisions knowing they can depend on you to manage the part you play with a particular methodology.

But what if these routines and protocols are holding back effective working practices? What if the solution is not clear and easily implemented change? What if they are not actually bad methods, but there are opportunities to improve?

If we accept that innovation is the application of a better solution to meet a need, whether articulated, new or unarticulated, we can consider the two main ways that can happen: Evolutionary type innovation where incremental improvements are made.

Revolutionary innovations, which are disruptive and new.

www.theconnectedclinician.com

The key question when looking at how practice can change to more effective methods is; what capacity does an organisation have for these kind of innovations?

The limits on capacity can come from any part of the community, and are many. It takes a brave individual to ignore the discontent that can emanate from trying something new. Particualrly of the revolutionary type of innovation.

As I read back through what I have written so far, I’m wondering where this train of thought is taking me. I guess really, I want to be innovative. I want the freedom to try out any idea that looks like there is some merit in it. Some communities have taken an overt and explicit approach to innovating in their community (check out SIS Hack) and I can’t remember where I heard it, but the phrase ‘The sacred cow, makes the tastiest burgers’ resonates when this kind of approach is taken.

As amazing and inspiring that kind of leading is, this isn’t happening for me. I need to find another way to innovate without disrupting the consistency in my context so much, that I get shut down. So I suppose it falls to me to be ‘innovative in innovating’. We’ll see how that goes.

Not 'just' another tool....

My parents just about remember writing in chalk, on blackboards. Then along came the pencil and paper and revolutionised learning. Well not really. It just let teachers and students do what they had always done, but better, faster, and more conveniently.

https://www.flickr.com/photos/thecampbells/3005650135

So how do we make sure that technology doesn't become another story of the pencil and paper, and just make things better, faster and more convenient?

To me it seems obvious. And yet so often I see technology employed to do pretty much exactly the same things as have always been done in class, but on a screen. And sometimes, even the 'electronic worksheet' makes an appearance, a more depressing use of technology I can't imagine.

I think, a clear indicator of how effectively you will use technology to support learning is how you facilitate learning without technology? Do you seek out imaginative methods of supporting learning? Do students have autonomy in your class, do they understand the bigger picture of what they are learning? Do your students look forward to coming to your class and enjoy the challenges of learning with you?

If these things are not happening at all in your current teaching and learning, then dumping a bunch of technology in your class is mainly going to be a large waste of everyone's time and money.

http://pr0tean.blogspot.co.uk/2013/08/transformational-technology-simple.html

If you don't want to waste time and money, have a read of this explanation of the SAMMS acronym. Match it up with the commonly referenced SAMR model of transforming learning with technology. Then reflect carefully on your practice. It's too important an opportunity to let your students down on.

Using technology to increase engagement?

Don't.

Simply put, it's a mistake. If you use technology purely to increase your students' engagement with their learning, then you are wasting your time, and more importantly, your learners' time.

Yes, it may happen that you can perceive an increase in students engagement when using technology, but that is not a reason to use it. This is because as the technology becomes embedded, it loses its initial excitement, the novelty factor, and becomes another facilitator of learning. Like a pencil. An incredibly powerful and effective pencil. But still like a tool to be used for learning. How many teachers claim to introduce the use of a pencil to increase student engagement? None, you'd be mad to. Unless pencils are used so infrequently that they then become a novelty, then they may work, briefly. 

Guess what? Technology such as iPads are not a novelty to students. So many have them, or have used them already that although they may be new to the teacher, they are not actually that special to the students. Being allowed to use them in school is a novelty. But that soon wears off if you are using them to achieve the same things that you have always achieved. If one of your main foci is to increase engagement, then there is the temptation to miss the opportunity to take learning to different levels. In fact, if engagement is a substantial issue for your class, then you probably need to take a hard look at how you are organising learning in your class anyway, never mind throwing technology into the mix.

The novelty of technology increases engagement for almost all students.  But as soon as the tasks on technology becomes schoolwork, then play becomes work.  If the students are not motivated by their schoolwork, it doesn't matter how it is completed, they'll still not be motivated.