ranking list

[awk]

I follow what you say, but maybe I didn't express myself very well, because as far as I can see, your response Graeme doesn't seem to say anything about what I was saying. (Or maybe it does, but I just can't see the relationship!).

See if I can put it another way: if X does really well in regional events, maybe even winning, then s/he will get a certain number of points that puts them in front of everybody else who ran that course. They may do this several times, and get enough points to put them near the top of the rankings.

Then our elite runner Y comes along, peaking for the big races. X still runs like they were compared to everybody else running the regional races, and will thus presumably get a similar number of points as previously. However Y beats them. Surely, then, Y will get even more points?

In other words, what I'm saying is that therefore, surely, the big races don't need to be weighted. Because the points are based on who is running and where they are ranked, if you beat a very ordinary field (more typical of a lowly regional event), you aren't actually going to score huge points. But if you beat a field with a very good field (more typical of the British) then you are going to score big points. You've got to beat the field at that lowly regional by a pretty big margin if you are going to get anything to match.

Or have I misunderstood how the system works?

Does what you say mean that what one perhaps should do if trying to find the best person, is count six scores as previously, but allow the British to count twice or three times over (so if it's your best score, it'll actually feature those times in your six counting races)?

[graeme]

The question is whether it's easier to beat a quality field by a small margin, or a weak field by a huge margin. The way the list is designed, each of these should be equally difficult to do. Your elite runner Y can collect big points by doing either thing, and will in either case be ranked above colour-coded king X.
So, you, I and the ranking WP agree you don't need to weight the big events.

The "problem" which others see is that if you run loads of events you get loads of chances to crush the weak field, and so its more likely that your big scores will come from one of those many little events. They think that, e.g. winning the British is such a reliable indicator that you're a good orienteer, you shouldn't need to prove it elsewhere.

If you hold the second opinion, then the correct statistical way to treat it is what you say in your last paragraph.

[Adventure Racer]

The problem with your theory, awk is that your ranking is based on your best 6 results, whilst your score used to generate the individual event rankings is based on the average of all your results. The people getting anomalous big scores will likely be running lots of events, therefore diluting the effect of those scores on the score used to generate event rankings.

Is this average of all results what you mean by "secret list", graeme?

As to making BOC or JK count 3 times, that's not really going to help a lot. If the person with big anomalous scores runs badly at these events it doesn't really matter how many times you want to include it. Meanwhile it's still unlikely to be one of the best scores for the winner of these events (or is it?)

[awk]

Thanks both - interesting point AR. It'll be equally interesting to see how it pans out. Like many innovations, it may not be perfect at introduction, it might even be far from perfect, but I'd like to see it given a chance to run for a while. Although you never know, there's still just time I think for somebody to put a resolution to the AGM to reverse it, and split it into all sorts of different classes again!

[The Loofa]

One thing I fell really bad about was never getting round to making a submission to the Ranking Working Party. Anyway – in an attempt to make up for it -

As I see it the new listing has 3 major faults

1) Omission of night events. I see no justification for this. A couple of reasons were offered further back up this thread (anomalous results, small classes) and they were swiftly debunked by the next poster – nuff said.

2) Not weighting the more prestigious events. Its Awk and Graeme who are wrong here. As I’ve explained before on a previous thread people run faster at a more important event (at least they are supposed to) and the weighting is needed to compensate for that. 4-5% on all L1 events should suffice.

3) Including too many less prestigious events. This is actually the inverse of the previous reason and has been raised already in this thread. By including old C4s or (what someone called) lower L2s, you are picking up events where a significant proportion of runners might not be trying too hard, hungover, injured, trying out something new and different.

2 and 3 are really the same reason and I raise it in order to improve the credibility of the list. Your ranking is your 6 highest scores, and for that to be credible should be your 6 best runs – not your 6 biggest flukes dependant on what others were doing.

A couple of other points.

- there was a line earlier – “outliers will be ignored if x% slower than the winner” (or something like that). But where the outlier is the winner, that’s asking for trouble. The Cat Taylor example being the obvious one here.
- In the old system you could find details of the calculation – showing each runner’s previous average and weighting this time. I haven’t been able to find it in the new one. Am I missing something?

[graeme]

Not weighting the more prestigious events. Its Awk and Graeme who are wrong here. As I’ve explained before on a previous thread people run faster at a more important event

You keep repeating this. And I keep asking for the evidence (of which there is none, AFAIK). So I'll ask again.

But even if everyone did run 5% faster at National event, they'd all still get exactly the same points as at a normal event. Statistically, multiplying points by 5% is just like multiplying the measured rainfall by 5% at the more accurate weather stations. Maybe this "global warming" is just because people have better measuring apparatus nowadays, and so are justified in multiplying up the temperatures

There is a problem with including small or less prestigous events, or events which people don't take seriously (night-O springs to mind ). There's even a statistically valid way to ameliorate it. Multiplying points from big events by a random number isn't it.

[distracted]

The Loofa -> The ranking list will never be a perfect indicator of ability relative to everyone - there are far too many factors involved that mean there are inherent differences between events and amongst competitors' performances at events.

The best 'ranking' indicator of ability is probably a mean/median score; I find the 'top 6' method of sorting rather arbitrary and more of a competition than a ranking.

If you think people run at their best at the national level events, where the majority of the best orienteers will be, then (as has already been said) the best way of showing this is to set up a separate competition based only on these events. This is how a new 'Masters Cup' or similar could be run, along the same lines as the UK Cup. Then, competitors truly have a head-to-head ranking over the best events of the year.

[andypat]

There is a problem with including small or less prestigous events, or events which people don't take seriously (night-O springs to mind ).

The interesting thing about this debate is the number of different arguments and points of view. For instance Graeme puts a solid argument based on his knowledge of stats for not including night events or weighting certain bigger events. You cant argue with that. But if part of the remit of the ranking list was to encourage more ranking events (see the RWG link somewhere above) then surely not putting night events on is counter productive, particularly if they are prestigious night events. Conversely by not allowing night events in the rankings surely this will result in further downgrading of night events with less people wanting to go etc.

[Simon]

As I’ve explained before on a previous thread people run faster at a more important event (at least they are supposed to)

Why am I supposed to run faster at a more important, as designated by the fixture list level, event? You cannot assume that the events people will deem important, and so plan to run their best at, will match up with the level the event has been given in the fixture list.

[The Loofa]

You keep repeating this. And I keep asking for the evidence (of which there is none, AFAIK). So I'll ask again.

Evidence
1) Let's start with the one I have quoted to you before and see if you remember. Colin Dickson in 2006. That year he won BOC and both days of the JK at M50. He was top of the ranking list - something no-one would dispute. BUT his 6 best scores did not include BOC or the JK. Do you really believe none of those 3 runs could have been in his best 6?

2) I'm sure most people can see examples for themselves where they didn't score as well as they thought they deserved at a (now) L1 event and/or collected a lot of points at a lowly event for what they thought was an average run.

3) Since turning M50 (2004) I've been running a spreadsheet of the top 40 (approx) in my age class and which events their scoring 6 come from. I get many examples of some events scarcely showing and some events featuring often. when I look further at the latter I invariably find several people scoring well below their running average thereby pushing others' scores upwards.
(which is why I finished my previous posting asking if there was any way this could be accessed under the new system like it was with the old)

Well - that's more than enough evidence for me.

[The Loofa]

But even if everyone did run 5% faster at National event, they'd all still get exactly the same points as at a normal event.

Of course. But if you run 5% faster than you did at the previous week's L2 don't you think you should get more points for you better run?

[The Loofa]

Multiplying points from big events by a random number isn't it.

It may not be easy to calculate; its certainly impossible to come up with something deadly accurate. And yes I do realise that 5% faster should not mean 5% more points.
But on ething I am sure of - its not random, and a reasonable estimate can be found

[The Loofa]

Graeme puts a solid argument based on his knowledge of stats for not including night events or weighting certain bigger events. You cant argue with that.

Really. Gosh - spoilt for choice with the emoticons here. Let's try

[Scott]

One suggestion that was made to the workgroup was that, instead of being applied to the total points scored by each competitor, a weighting could be applied to the distance from the mean points of each competitor, so that people who did well at National events got bonus points, but people who did badly got fewer points than they would at an L2. That way the mean points would stay that same.

I have a suspicion that this only works if you calculate the points by class rather than by course (graeme may correct me here), but that might not be an problem at National events anyway.

[The Loofa]

Why am I supposed to run faster at a more important, as designated by the fixture list level, event? You cannot assume that the events people will deem important, and so plan to run their best at, will match up with the level the event has been given in the fixture list.

You can't assume it for everyone, but I think you can expect (and would find**) a good degree of correlation.
** as orienteering doesn't go round a 400m track or some other precisely measured distance, but is different every for every race (and that must be one of the main reasons we all love it), it makes that "finding" damnably difficult