British Revised Extended International Table of Qualifications

As you may have heard over the weekend, the DfE has today (1/4/19) announced a new key performance measure for schools in England.  All schools in England will be reporting on the new measure for current year 11 students.

The new measure is similar to the Ebacc, in that it combines the grades from a set of subjects and the school is measured by the proportion of students studying that set of subjects.

The new British Extended International Table of Qualifications measure will also be awarded to students at a ‘stable’ or ‘strong’ level if they have achieved at least a ‘stable’ (equivalent to level 4) or a ‘strong’ pass in each of the subjects.

Subject included in the British Revised Extended International Table of Qualifications

  • English
  • Maths
  • English Maths
  • Science (there will be a remodelling of science to remove all the Greek letters from equations, but that is not anticipated to be implemented at least until the exams in 2020)
  • Languages (Welsh, Irish, Northern)
  • Geography
  • History (Shaping the Nation and WWI and WWII modules only)
  • Drama (French, Spanish and German will be considered part of the Drama curriculum as the key skills for those subjects is to talk loudly and slowly)

These subjects are divided into Hard and Soft categories.  To achieve a Hard British Extended International Table of Qualifications you must achieve as strong pass in English Maths, History and Drama.  Achieving stable passes in Geography, English and languages will modify the qualification to a ’Soft’ British Extended International Table of Qualifications.

I realise this may cause some confusion as it has been introduced so rapidly, and seemingly without being properly thought through, and the implications may be quite far reaching for our curriculum.  I understand that Ofsted will not be using this measure until Brexit has been completed.

 

Advertisements

How to set grade boundaries in the new number grade world


Setting grade boundaries is very difficult. In reality, even the actual published grade boundaries for a specific exam paper may not really be valid for a different cohort.

Add in the changes to both the content and the grading systems for most GCSEs, and we have a very complex situation.

Some important points

Internal consistency

If the marking is accurate and has been moderated, it should be internally consistent. I.e. the students who gets the most marks has the best attainment.

This means you will always be able to compare the attainment of individuals and groups within the subject.

External pegging

The problem comes when you try to link a set of marks to an actual external real life GCSE (or other) grade.

There are some assumptions we can make that will help

  1. We know the grade equivalencies. The bottom of a grade 4 is the same as the bottom of a grade C.  the bottom of a 7 is the same as the bottom of an A, and the bottom of a G is the same as the bottom of a 1.
  2. Equivalent outcomes – grade boundaries are being set so similar proportions of students achieve each grade as in previous years
  3. Our marking is internally consistent
  4. Our students are likely to achieve similar outcomes to previous years.

Methods

Ranking

Put all the grades in descending order

Find the attainment percentages for last year in your subject (https://schoolsweek.co.uk/gcse-results-2017-uk-subject-tables/ for national data)

Use the previous attainment data to work out how many students are likely to achieve an A grade or better.

Count down from the top until you reach that number

The mark that student achieved is a good candidate mark for the Grade 7 boundary.

Repeat for the number of student achieving a C or better, to get the Grade 4 boundary.

Repeat again for the number of students achieving a grade, this would be the Grade 1 boundary.

You can then look at individual grade proportions to split out the 2,3,5,6 and 9 boundaries, or make a logical spread throughout the marks.  e.g. if you find that the Grade 4 was at 30 marks and the Grade 7 was at 45 marks, you could put the 6 in at 40 and the Grade 5 at 35.

Some points to consider

This is not very far removed from how the exam boards will set their grades.  They will just have a lot more numbers to play with.

There is no such thing as a completely accurate grade boundary. The best you can do is get a good idea of where a student is compared to the rest of the school and to the exact exam paper.

Grades generated this way would be projections, ie the grade that the test indicates they will achieve in the real exam.  To convert to an actual Working At Grade you will need to raise the boundaries slightly.

 

Use old style exam boundaries

The old exam boundaries are likely to be significantly higher than the new ones due to the changes in challenge presented by the new courses.

For example, in Edexcel Mathematics

Grade 2016 2017
Higher Foundation Higher Foundation
A/7 70% 52%
C/4 35% 71% 17% 51%
G/1 31% 11%

 

English is not as easy to compare as that has shifted from tiered to a single paper, but by looking at the single non-tiered paper 3;

Grade 2016 2017
A/7 90% 65%
C/4 59% 45%
G/1 13% 10%

 

Typically, the percentage of marks needed to achieve a grade have shifted down by over 15 percentage point at most grades.

This is a huge difference compared to the normal drift of grade boundaries shifting each year.  For maths, this has been in a range of about 6 percentage points (eg a C in maths has had a range from 29% to 35% from 2014 to 2016).

This being the case, shifting grade boundaries down by something in the region of 10 percentage points would not be unreasonable.

Make up grade boundaries based on gut feeling

Don’t

In some cases, this may seem like all you can do, but in reality you will effectively be using a ranking system without necessarily doing it formally.  The advantage of doing it formally is that you can discuss how you got to the grade without having to say you made it up.

Hybrid

  1. Set grade boundaries using an adjustment (+/- fudge factor) on previous grade boundaries.
  2. Rank order the students and see if the grades look right; ie the students are getting the sort of grades you expect based on their work in lesson.
  3. Adjust the boundaries until the results look right. NB this is not adjust the boundaries until the results look good
  4. Review the boundaries and makes sure they are not too far away from what has happened previously

 

A hybrid method is probably the best approach as it makes use of all the sources of information and combines them to give you something consistent and explicable.

 

Worked Example for Physics

Old Style GCSE grade boundaries (converted to number grades)

Grade Old GCSE %
8 75
7 60
6 45
5 38
4 31
3 17

 

We are pretty sure they are too high, but how much too high?

An adjustment of 10% seems reasonable based on the maths data

Grade Old GCSE % Adjusted
8 75 65
7 60 50
6 45 35
5 38 28
4 31 21
3 17 7

 

 

So looking at historical grade distribution for our school (we want to do better than this, but it’s a good place to start)

We should have in the region of

Historical % of students achieving that grade Number of Students in current cohort
8 10 4
7 22 9
6 30 13
4 28 12
3 10 4

 

Running through the results in order, the top 4 students should get an 8.  The lowest mark from the top 4 students was 60%.  So that makes a good grade boundary for the grade 8.  Then another 9 students takes us to 42%, so a good boundary for the 7.  And so on to find suitable boundaries for the rest of the grades.

Grade Old GCSE based on similar proportions
8 75 60
7 60 40
6 45 33
5 38 30
4 31 20
3 17 17

 

Taking all those together we can derive an reasonable ‘final’ grade boundary based on both approaches.

Grade Old GCSE based on similar proportions Adjusted Old GCSE Final
8 75 60 65 65
7 60 40 50 45
6 45 33 35 37
5 38 30 28 33
4 31 20 21 25
3 17 17 7 15

 

Plugging these into the class results gives a set of data that looks about right, i.e. the distribution is slightly worse than students achieved last year, so we are probably not making it too easy, but they are not so much worse that we can’t work out who to target.

It is important to remember the point behind grading tests at this stage – we need to work out who we need to work with to get help them achieve their potential.

Are selective schools the end of civilisation as we know it?

First of all, I need to be very clear here, I am horrified that the government is even considering going back 50 year and reintroducing selection.  It goes against every instinct and makes me angry.

That being said there may actually be a way of making it work.

I first started teaching in grammar schools and then moved to the non-selective schools around them, so I have some experience to draw on here, but I am drawing in quite broad strokes with a healthy chunk of generalisation.

What are the advantages of going to grammar school?  As far as I can see they are

  • Better resources – well off areas parents will chuck money at the PTA as it is a cheap alternative to private school
  • Better behaviour – as a rule the kids who work hard to pass the 11+ are not the kids who kick off in lessons, for a huge number of reasons
  • Umm, that’s about it.

Any upgrades in outcomes will tend to flow from the first two points – basically if you can chuck time (less disruption in lessons) and money (more funds to buy cool stuff) at a problem (exam results) the problem gets solved.  Happier teachers, less disruption, students who are used to working and or have parental support makes a winning formula.

At the moment in selective areas grammar schools can outbid anyone else to get specialist teachers in every class.  These teachers don’t tend to be very mobile once in post and will continue to get a good set of grades year after year.

This may not be the best way of making sure that the kids are getting the best teaching.  It may be better than the kids get in the non selective school as those schools are a much more difficult prospect.

It is far more difficult to be effective in the non-selective school because

  • The kids have already been told they are failures
  • There is an increased concentration of difficult students

This leads to rapid staff turnover, which leads to worse outcomes, which leads to staff giving up and leaving and round we go again.

I have to say I am looking forward to seeing Progress 8 based performance tables.

But now, a possible route to make selection work for everybody, and it makes use of the free market economics so beloved of our Tory overlords.

We need to stop the selective schools taking all the best teachers and all the spare cash.

With the collapse of national pay scales and the rise of MATs we can make sure we get the best teachers in the non-selective schools by the simple expedient of paying them more.

Give them all the equivalent of London weighting and you will get staff that want to work there, which means you can recruit better teachers.   If you make selective and non-selective schools work in pairs (or in MATs) you can balance out the pay structure so you have the choice between easy and lower pay scale (selective) or bloody hard work and higher scale (non selective).

Make any PTA/alumini/other donations to the school (well, MAT) go to both,  even if you think you are donating to your posh old boys network.

Take it further – make MATs work locally and have a network of different types of schools for different types of pupil – lets have more UTC type schools in the mix.

You know what, all that is achieving is segregating by class.  But it’s a better idea than just introducing selection for any (all?) schools that want it.

Computer Science reaching for the top of the taxonomy

After a year of feeling like I am bashing my head against a wall… a moment of realisation.

I have been trying to teach new spec GCSE computer science to a group of students who are really struggling.  This has of course led to lessons that are frankly hell on Earth for all of us.  The students are stuck and thus misbehaving, but I have not been able to find the simplification to make it workable.

This afternoon I looked at a slightly different presentation of Blooms taxonomy than the one I had by my desk for years.  Queue Giraffe Moment1.

The top of the tree, the highest level of thought and abstraction is ‘creating’.  This is what we aspire to achieve with our best students, as we move them towards more abstract and independent thought.

Unfortunately for me and my students, it is also the basis of the programming aspects of computer science.

You cannot program unless you can turn a concrete description of a process into a highly abstract representation of that process. You need to assemble/construct/create/design/develop/write an algorithm.

This is a huge ask for anybody, but for a fourteen-year-old who is struggling to get to grips with maths and English it is possibly a step beyond.   Add in the requirement for resilience to deal with the whole ‘fail better’ approach to developing programs and you have a situation that can lead to failing totally.

From where I am sitting there is not much concrete to build up to that creative level from.

Just to take a quick look at learning to write computer programs from a Bloom’s point of view.

Remembering – being able to define key terms such as sequence, selection, iteration, assignment, variables, conditions and so on (probably not many more than that to be honest)

Understanding – being able to explain what the purpose is of those constructs and why you would use them. Being able to understand an algorithm that has been given to you.

Applying – choosing different constructs to solve specific problems e.g. – would you use a for loop or a while loop in this situation.

Analyse – be able to identify different approaches used in an algorithm

Evaluate – be able to justify a choice of approaches used in an algorithm

And then Create as mentioned above

In what other subject are you expected as pretty much the fundamental task to be able to create something as complex as an algorithm to solve a problem.  I can only think of composition in music as any sort of comparison, and even then if its not good it still does something.

I have read previously about the problem of shifting students from pile of Lego stage (knowing there are these different constructs that you can put together to solve problems, see https://wordpress.com/read/feeds/1141878/posts/936233556 )to the creative stage, but it had not sunk in why it was so difficult to make that leap.

Teaching students how to write a loop or another construct is fairly straight forward.  It is just learning a formula that you can apply.  Then we ask students to take their toolkit and try and solve big problems (e.g. write a program to generate a sequence of random maths questions, store the results in a file, have separate files for each class, keep only the top 3 grades per student and be able to present the results alphabetically, by score and by average score…from the 2015 controlled assessment)

If we could just work with understanding algorithms and programs we would be able to build up ideas and go somewhere.  Unfortunately, this is not enough for the computer science GCSE.  With a 20-hour practical exam (the practical exam from hell­2) to fit in and mark before May, that is at least one term down.  So you are left with about 40 weeks to learn to program, oh and to study and learn the other 66% of the content for the exam.

So how are we going to achieve enlightenment?  At the moment I am stuck.  I really don’t think ignoring the non-examined component is any sort of viable route, but I am also really struggling to see a route from concrete to creative and resilient.  A plan is required and anybody who has a good idea… please let me know.

 

1 Giraffe Moment needs some explanation.

Everybody knows what a giraffe looks like.  When you were little you were probably surrounded with cuddly giraffes, cartoon giraffes (and elephants, and lions and tigers and…).   You are so used to the image of a giraffe you have forgotten how utterly strange they are.  Go to the zoo and look at one with fresh eyes.  Look at that neck, the tongue, the mushroom shaped horns.  What a weird beast.  This happens a lot, something that is amazing becomes commonplace and you stop noticing what it really is.  The giraffe moment is when you look at again and realise that it is seriously weird or interesting and it is ripe for a re-examination.

2 The practical exam from hell

A substantial chunk of the new GCSE is the non-examined component, which simply put consists of giving a student a problem, and then telling them to get on and solve it.  They have 20 hours in exam conditions and no access to anything apart from the offline help files in the IDE (if any).  So no help from books, internet, teacher or peers.  That is quite scary for me as a fairly competent programmer, for a slightly lost student it is an absolute terror.

Reconciling with the knowledge based curriculum

For the last few months I have been struggling to remind myself why I want to teach, and what education really is (see ‘remembering stuff as education’).

I think I may have found my way through.

The remembering stuff is the end effect, not the point, of education.

Headguruteacher (https://wordpress.com/read/feeds/2402438/posts/1020119340) has it right when he talks about the mistakes in the mental model.

When it comes down to it, the mental model is the knowledge we want students to have. It is the model that we need to be learning by rote, not the answers.

The word equations for respiration, photosynthesis and combustion are all coming from the same root model – using oxygen to break bonds and release energy. If we have learnt that model, then rest is easy.

There has been a tendency in Physics textbooks to present every derived version of a formula as a whole new chunk of knowledge to be leant. This is a bad idea.  It makes pattern matching easier, and removes the need for understanding.

I struggle a little more to apply this outside of my fields of expertise (how useful is it to learn by rote ‘range of causes of the Renaissance, the principal features and their outcomes’ in year 7, and what sort of mental model you could use to make that easier is beyond me).

So with my thinking reset, the controlled assessment winging its way to the examiner and the sun shining…..time to start working out new ways of making this stuff work.

What does expected progress actually mean?

The arrival of Progress 8 may well have revealed a massive hole in the way we measure students’ progress.

For the past twenty years we have been expecting all students to make 3 levels of progress from KS2 to KS3. This has gradually been stretched to the top students being expected to achieve 4 levels of progress.

At least one academy chain has set a target of 80% of students achieving 3 levels of progress, and 40% achieving 4 levels.

This is all very nice and expects students to do really well.

But what do we mean by ‘expected’

A quick dictionary definition

regard (something) as likely to happen.

So the likely outcome for any given student is to make 3 levels of progress.

Some will make more, some will make less.

80% of students making that level is a pretty high hurdle, but most students achieving what is likely to happen. I can go with that.

40% achieving more than the most likely. That starts looking quite challenging, but let’s go for it.

But hold on, where did this 3 levels of progress being likely come from?

I assume that it was built into the design of levels and GCSE grades, the research took years and costs millions to get those levels set up, so one would hope that they do the job.

Then a big spanner falls into the complexities of guessing how much progress students should make.

Progress8 has shown us the progress that students do actually make (at least the average of the average of their grades with some complexities about what counts where)

What does that tell us then?

 

Average fine level 2015 Attainment 8 estimate Average levels of progress
1.5 13.01 2.0
2 17.57 1.8
2.5 19.14 1.4
2.8 20.11 1.3
2.9 21.62 1.3
3.0 23.08 1.3
3.1 23.65 1.2
3.2 24.16 1.3
3.3 25.88 1.3
3.4 27.42 1.3
3.5 28.27 1.3
3.6 29.95 1.4
3.7 31.81 1.5
3.8 33.33 1.5
3.9 34.98 1.6
4.0 36.12 1.7
4.1 38.45 1.7
4.2 40.30 1.8
4.3 42.24 1.9
4.4 44.15 2.0
4.5 45.52 2.1
4.6 48.20 2.3
4.7 50.43 2.4
4.8 52.48 2.5
4.9 54.77 2.6
5.0 55.88 2.7
5.1 59.07 2.9
5.2 61.28 3.0
5.3 63.70 3.1
5.4 66.12 3.3
5.5 65.46 3.5
5.6 71.42 3.6
5.7 74.00 3.8
5.8 76.14 3.8

 

 

Only those students achieving an average level of a 5 or more at KS2 are making anything like 3 levels of progress on average.

That’s only the HAPs (high Achieving Pupils) making expected progress, only the HAPs achieving ‘what is likely’.

The average student (level 4 at KS2) only makes an average of 2 levels of progress.

Now the same calculations using the recently published (but soon to be abandoned) Value Added coefficients.

 

KS2 Average fine level Levels of Progress
P8 VA
3.1 1.5 1.3
3.2 1.4 1.3
3.3 1.5 1.4
3.4 1.5 1.5
3.5 1.5 1.5
3.6 1.6 1.6
3.7 1.7 1.7
3.8 1.7 1.8
3.9 1.7 1.9
4 1.8 1.9
4.1 1.9 2.0
4.2 1.9 2.0
4.3 2.0 2.1
4.4 2.1 2.2
4.5 2.1 2.2
4.6 2.2 2.3
4.7 2.3 2.4
4.8 2.4 2.5
4.9 2.5 2.5
5 2.6 2.6
5.1 2.8 2.7
5.2 2.9 2.9
5.3 3.0 3.0
5.4 3.2 3.1
5.5 3.0 3.0

Oh look, nowhere near 3 levels of progress unless you start on a level 5.

It’s all very nice to have high expectations of our students, and try to push them top achieve, but this looks more and more like those in charge of education not understanding numbers, they want everybody to be above average.

Talking of which…

A recent discussion with an OFSTED inspector indicates that a P8 score of 0 is always going to be ‘Requires Improvement’. This means every school that is average or below is in trouble.  That is going to be around 50% of schools.

How long is it going to take before Ofsted inspectors get their collective heads around these changes, expecting the average progress of students to be significantly above average makes a mockery of the whole system.

 

DFE replacement for Levels announced

The DFE have finally announced their alternative to levels.

These will be the measures used to track progress from summer 2016 onwards, so we really need to make sure we can back translate our progress from our current system to the new system.

The new system is based on a student’s progress through Age Related Expectations.

Age Related Expectations are given a rating for each year group, and for each subject (see table below)

So a student achieving Year 7 ARE in English will report that grade, whatever chronological school year they are actually in.

For progress grades go in single steps from the colour Magenta to the smell of freshly baked baps.

Attainment grades go in half steps (ie each step is worth half a point of progress) from Blancmange to the sound of a cantankerous Armadillo.

So to take a more definite example

A student in year 8, and has achieved above expectation in Maths. This student is therefore a small green llama, but may only have achieved ‘that slight feeling of nervousness caused by the visit of a distant relative’ in Maths.  Therefore they are not yet making ARE.

I think you will agree this is a far more useful system than has existed previously.