Ranking List

[Spookster]

Just seen that start times and heats at the British Sprints will depend on rankings, so some things are starting to rely on it. Oh dear.

I've been using the ranking list for years to do start list seeding. It's fine, as long as you don't only use the ranking list, and make sure you spot anomalies like very good runners with fewer than 6 scores.

[graeme]

Just seen that start times and heats at the British Sprints will depend on rankings, so some things are starting to rely on it. Oh dear.

What would you propose to use? If you look at how unfair some of the junior course were last year (no ranking to use) you'll see how bad randomising can be (e.g. in M14 EVERYONE in one heat was over a minute faster than qualifiers from another heat - there were small differences in courses, but nothing to explain that).

[greywolf]

I've been using the ranking list for years to do start list seeding. It's fine, as long as you don't only use the ranking list, and make sure you spot anomalies like very good runners with fewer than 6 scores.

unfortunately the rules for the sprint champs say (in bold) "For a particular age class each heat shall, as far as possible, be equal in the number and quality of the runners. The Rankings scheme shall be used to achieve this, except for the junior age classes where the Events and Competitions Committee will provide a seeded list."
which doesn't appear to give any option to correct anomalies...

[AndyC]

Yes, but all sensible organisers are going to adjust in the way Spookster suggests.

[graeme]

The rankings "scheme" provides an "average score" for each person. This is different from the published ranking and contains just the information spookster wants to use. e.g scotia (ranked 775th) has the highest average score, and even GG (based on only one run) is a seedable top ten rather than 4360th.

One curious thing about non-elite seeding is that in the Sprint/middle its advantageous to be seeded (you get a weaker heat), while in BOC/JK it's a disadvantage (less chance to run with someone good.

[awk]

I'm not bothered about the outcome of the ranking list being used - I'm bothered about the fact that the rankings list is being relied on in competitions. Thin end of the wedge.....

As the published lists stand, I don't think it's accurate enough to do what it sets out to, which is decide who are the best in age classes and who are not. Greywolf said as much when he commented on the margins or error: if the margins of error are sufficiently great to be 8-10 places inaccurate, then it's not accurate or reliable enough for the job it's being used for.

Now if it's using the hidden average, this may be more reliable(I suspect it will be); equally the adjusted scoring may, eventually, produce a more reliable result, but the one that is published is, IMO, not. I know others will disagree, and I've said more than enough on the subject....!

[roadrunner]

I've been using the ranking list for years to do start list seeding. It's fine, as long as you don't only use the ranking list, and make sure you spot anomalies like very good runners with fewer than 6 scores.

unfortunately the rules for the sprint champs say (in bold) "For a particular age class each heat shall, as far as possible, be equal in the number and quality of the runners. The Rankings scheme shall be used to achieve this, except for the junior age classes where the Events and Competitions Committee will provide a seeded list."
which doesn't appear to give any option to correct anomalies...

Saying the ranking scheme "shall be used" isn't the same as saying you seed people purely by reference to ranking position; to me, it means you use the information the ranking list provides to decide who should be seeded - which is exactly what Spookster say he's done before. Using the average score when someone has less than six scores seems to me a good approach, it's a shame that it requires significant manual effort when the information could be made more readily available.

[Sloop]

I completely disagree about using the hidden average for major competitions. I am not the most reliable orienteer, and have in general been improving steadily over recent years. So my hidden average is almost certainly misleadingly low, and my top-6 is very close to a real indication of how good I can be when things go well.

I know that in the past, junior selections were based on how good an athlete was occasionally, rather than what their typical performance was - which is the same argument.

[roadrunner]

I completely disagree about using the hidden average for major competitions. I am not the most reliable orienteer, and have in general been improving steadily over recent years. So my hidden average is almost certainly misleadingly low, and my top-6 is very close to a real indication of how good I can be when things go well.

I know that in the past, junior selections were based on how good an athlete was occasionally, rather than what their typical performance was - which is the same argument.

You have a "top 6", and I don't think anyone said that the current ranking points wasn't a good guide in this case. But for someone who's run less than 6 events, then surely the best you can do is the average of the points for the 1 to 5 events they've done?

[NeilC]

unfortunately the rules for the sprint champs say.....

As someone who was heavily involved in both devising the current sprint championship format, and for writing the rules I'd like to make the following points:

i) BOF rules were written by volunteers and should not be treated as watertight legal documents. Often we knew what we wanted to say but weren't able to say it succinctly enough, and as a result a rule can be interpreted more strictly then was intended. In many cases common sense can be applied to apply the spirit rather than the letter of the rule. Some will argue that the rules need to be absolutely clear, and that would be the ideal situation. Well at least now everyone has a chance to comment on the draft version of the 2014 rules (scroll down BOF website home page for details on how to).

ii) When the new format was devised it was done so knowing that the combined ranking list would be very useful in producing heats of equal ability. The intention was not that only the list should be used. When I organised the event in 2011 the heats were done in three ways
a) The open/elite classes were seeded by someone like Spookster based on all available evidence
b) The junior classes were done by someone else (from junior competitions group?) based on whatever information they had
c) The remaining senior classes were done by us based on ranking points but with myself and a few others having a check for obvious anomalies

[graeme]

I already posted on how badly the juniors got stuffed up, when seeders had to work without a ranking list. The senior seeding seemed to be quite fair, so ...

c) The remaining senior classes were done by us based on ranking points but with myself and a few others having a check for obvious anomalies

... who did you find with six ranking scores, who you thought was anomalous enough to seed/unseed?

[NeilC]

All we did was was list the entrants in age class in ranking list order and then allocate positions 1,4,7 etc to heat 1; 2,5,8 etc to heat 2; 3,6,9 etc to heat 3. If prior to this allocation one of us spotted a entrant lower down the list than expected they could be moved up based on their average ranking score. I don't actually recall actually doing this to anyone though. For the heats we gave people a choice of start times so didn't automatically have the best ones going off last. For sprints there is no minimum separation (greater than the 1 minute start interval anyway) between seeds.

[Paulo]

The ranking points have all changed again. Or at least those in the last 12 months appear to have done so.

From what I can tell, points older than 12 months ago are still based on the adjustment carried out a few weeks ago, which seemed to be roughly New Points = (0.8 x Old Points) + 296. However, points in the last 12 months seem to be something like (0.8 x Old Points) + 325. So all scores in the last 12 months are now about 29 points higher than they were when the previous adjustment was made a few weeks ago.

I still can't quite see the logic of how this change will correct the inherent problems in the ranking list, i.e. gradual downward drift over time and over-ranking of certain runners due to the initial seeding data, as this latest adjustment (like the previous one) seems to affect every runner's scores in the same way (using the formula above) and so the relative positions of everyone in the list now are essentially the same as they were before these adjustments made (any movements up/down the list this week probably being due to old events dropping off the list/new events coming on).

I'd be interested if someone (DJM?) could explain the logic behind the changes which have been made...

[madmike]

I'd be interested if someone (DJM?) could explain the logic behind the changes which have been made...

Maybe Spookster would be better not DJM's responsibility

[Spookster]

I'd be interested if someone (DJM?) could explain the logic behind the changes which have been made...

Maybe Spookster would be better not DJM's responsibility

No, the opposite. DJM is the member of E&CC with responsibility for rankings/rules, and (I believe) has been involved in making proposals for how to rectify/improve the system/data.