Comprehensively wtf?

Direct from a school website (green circle is mine):

Looks simple enough - but....
  • The title says [Last Year] - when exactly was last year? From the rest of the site, it becomes clear that Last Year is actually 2013, as the chart was put online in 2014 - so the title is wrong
  • The title says ..."with drill through to registration group" - no, this is not present in any way. And, no, the chart was not interactive.
  • What does the blue / pink split refer to - it's not on the chart?
After some serious poking on the website, it appears that the pink refers to "Pupil Premium" learners.
  • Ok, so there are 36+57+55 = 148 of those in total - with 57 in Year 1 (we'll come back to that in a minute)
  • There are also 116+148+147+149+36+27+55 = 708 learners in total
Is this visualisation helpful to a parent? How does this compare either to last year or local / national averages - without context this is a fairly arbitrary and meaningless piece of info.
Gender splits..
  • The same blue / pink split - so we conclude 73+75 = 148 are the Pupil Preimum learners - this is consistent with the previous chart.
  • 227+73+283+75 = 708, so the same total as before
  • Using a stereotypical colour coding blue/pink is normally used for Male / Female -- but in this case it's for PP / NonPP. Confusing
Again, is this male / female mix representative of local / national trends? Is it helpful to a parent?
I have no idea what this is actually showing, or what it's meant to, so won't offer comment other than Aargh!!!!
A table of the breakdown of the attainment of learners in Year 1, broken into subjects and whether or not the learner was a Pupil Premium recipient or not (I assume)
  • Lots of "noise" - confusing to read / compare -- what am I meant to be doing with this table? Comparing subject to subject or pupil premium to not pupil premium.
  • It says "59" learners got PP in Year 1 - but the previous chart (remember my green circle) showed this as 57 -- a small difference, but in means that I've got no faith in the data / visualisation.
  • Why is P4 below W? In fact as a parent, what's all this P stuff anyway? W?
  • The titles of the chart make no sense at all - they seem to imply that this is data for pupil premium learners - but then breaks the data into PP and Non PP
  • The total for each subject is 146, 148 and 149 - now previously we were told Year 1 had 147+57 pupil premium learners - I'm left very confused by this - and end up thinking that there are 147 learners in total and of that, 57 are pupil premium -- so the first chart is wrong.
Confused - and as a parent I'm not sure what I'm supposed to be inferring from these charts -- is the school doing well? Are PP learners being supported? - A genuine WTF moment!
If we are looking to visualise the impact of PP vs Non-PP, we can calculate a cohort average level (from the above data) and plot:
Now I can have a reasoned discussion about the impact (or not) of PP:
  • PP levels in Maths are approximately 2 sub levels above Non-PP
  • Non-PP levels in Reading are approximately 2 sub levels above PP
  • Writing - the impact of PP is negligible.
What does this mean though? Does it mean that PP is making a difference (as in Maths) or does it mean that PP pupils have multiple measures of deprivation and so do less well (as in Reading) and that the PP payment is not making a difference? Or as in Writing -- PP seems negligible - is this because of some wonderful PP intervention or because being classed as PP makes no difference here? Would a parent understand / appreciate these nuances? Is there any point in plotting all this?

Call to action
  • What is the purpose for displaying data on a website?
  • Does it tell a story?
  • Do you understand it?
  • Would a parent?
  • Is the visualisation correct?

comments powered by Disqus