Ranking List

[graeme]

I'm afraid mhister troll is right.
The rescaling hasn't changed anything - look for example at how many people are scoring last year's JK rather than this year's. They need to do what they said they were doing and restart the whole scheme (which should also get rid of the bias towards old men).
But you should remember that the scheme is required not to downgrade anyone due to a bad run - the published scores are not the even ranking system's best guess at who's best. The average score (which isn't published, but is used to drive the statistics) gives an even better list (with Scott Fraser at the top).

Even so, the ranking list top-10 is a lot more accurate measure of who's good than the JK top-10, the BEOC top-10, the UK-Cup top-10 or the UKOL top-10.

[rob f]

Well put graeme. While scores aren't perfect you're still roughly in the same place as people of the same ability.

[awk]

Well put graeme. While scores aren't perfect you're still roughly in the same place as people of the same ability.

Very roughly, and surely a ranking list should do better than 'roughly'. For instance, I am currently ranked 32nd in M55. Looking at the 10 M55s ranked in front of me, I have beaten 7 of them more often than they have beaten me in the past year, five of them by a margin of 4 events or more, the other two by 4-1 and 3-1 (using Ollie's website). One of the others I have never run against, whilst the two who have beaten me more often (3-5 and 0-1) are at the far end of those 10, ranked 74 and 76 places higher respectively overall, 8 and 9 places higher in M55. (The M55 ranked one place behind me has beaten me more often than I've beaten him, but otherwise nobody has as far as I can make out; beyond those ten in front, the scores swing the other way!).
Now that's a single situation, and there may be specific reasons, but that doesn't strike me as ranking me remotely accurately against my peers, and if it can't do that, I have little faith in my more global ranking. I had hoped this reset would do something about it, but it hasn't.
Having said all that, it's what happens in the forest that matters, so believe it or not, I'm actually not that worked up about it, especially if nothing of any importance is based on it. But if we are going to have a 'ranking' list, then it needs to have some sort of credibility, and those are some of the reasons why it doesn't, at least for me.

BTW, I think that idea of knocking off the extreme score is a good idea KP - but I'd go for 3rd to 8th results.

[roadrunner]

Don't know why they've done this with the rankings, but the relationship between old and new scores is linear and it is roughly

new = 0.8 * old + 298

old scores of zero are now being given as 298.

The percentage increase of an old 660 score is 25%
The percentage increase of an old 1000 score is roughly 10%

This is plainly wrong. If someone got 0 before, I think that means that their computed score was 0 or negative, and as negative scores aren't allowed it was rounded up to 0. So if a run that actually scored 0 is now re-scored as 298, then one that was previously scored -100 (rounded up to 0) should now be 198, not 298. Given the results are stored on a computer, and we're talking of an algorithm rather than manual processing, surely the effort (and time) to re-run all the results is minimal, so why take a short cut?

[AndyC]

I certainly can't see any scores less than 298 in the database (but I haven't checked them all)

[greywolf]

BTW, I think that idea of knocking off the extreme score is a good idea KP - but I'd go for 3rd to 8th results.

This seems like a good idea but would amplify one of the shortcomings of the system, which is that not everyone has the required 6 counting scores to get properly ranked. Increase the requirement to 8 and you just drop loads of folk massively down the list, which almost certainly outweighs the benefit of removing the odd anomalously high score.

Currently there are ~10 individuals who would probably be in the top 100 who don't have 6 counting scores (e.g. Scott Fraser, Graham Gristwood, Rhodri Buffett, Ed Nash, Matt Halliday, Ralph Street, Neil Northrop, Dave Godfree, Joe Mercer, Robert Gardner) - 34th on the ranking list doesn't mean you're the 34th best orienteer, just the 34th highest ranked of those with 6 or more scores...

For instance, I am currently ranked 32nd in M55. Looking at the 10 M55s ranked in front of me...that doesn't strike me as ranking me remotely accurately against my peers, and if it can't do that, I have little faith in my more global ranking.

With those you compete against regularly, you don't say whether their counting scores are from events where they beat you, or from events you didn't attend. Maybe it's frustrating to be ranked behind those you think you are better than, but the margins involved here are tiny - the M55 ranked 6 places ahead of you has only 50 more points on aggregate - ~8 points per event. What level of precision do you think should be acheiveable for a simple algorithm working with data coming from a sport with as many variables as orienteering?
Also, you say it's not remotely accurate when it ranks you as 32nd out of 300 or so M55s (with 6 scores), when you think you should be 26th... Yes it's not perfect, yes, BOF appear to have made an arse of the reboot, but at the same time folk need to have realistic expectations of the limits of precision and some basic understanding of the margins for error

[graeme]

For instance, I

I see your counting scores include more 2013 than most - you must be getting fitter!

The lazy way they did the rescale means the bias towards old results is still there - so you'll move up when those scores start dropping off other people's lists.
(plus what greywolf said - you wouldn't go round saying that a course was obviously flawed just because someone who you normally beat finished ahead of you).

[awk]

Fair comments both Greywolf and Graeme, and I accept all your comments about margins of error etc. If it was an unusual or occasional situation that would be OK. However, the rankings list is consistently like that, and that failure to reflect what is happening in the forest (e.g. I'm regularly beating those ranked above me, far more often they they are beating me) is why I don't have confidence in it. What I see as needing to be done as a minimum now is rerunning the reset so that it deals with the downward drift now and not in a year or so's time; dealing with the exceptional results, especially those from very minor races; dealing with the fact that some people's rankings are based almost entirely on events from almost a year ago.
I'm just thankful it's not actually used for anything important.

(BTW Greywolf, almost all races I didn't attend).

[DeerTick]

Perhaps we could take something from the world golf ranking methodology. The have a two year rolling system rather than our one. However, points scored in an event gradually diminish over that two year period :

For the first 13 weeks after an event the player receives the full ranking points earned in that event. However from then onwards they are reduced in equal weekly increments over the remainder of a two-year period. This gives priority to recent form. Each week the ranking points are reduced by a factor of 1/92 (approximately 1.09%) so that in week 14 only 98.91% of the ranking points are credited, continuing until week 104 when only 1.09% is credited. From week 105 the ranking points are completely lost.

(from http://en.wikipedia.org/wiki/Official_World_Golf_Ranking )

AP

[m4rk]

I like the idea from golf - but seeing as we can't implement the current system properly

The current scores of all ranked runners are rebased after each event to ensure that the
mean current score of all ranked runners is 1000 and the standard deviation of their
current scores is 200. This prevents the scores drifting over time.

then I don't think a more complex system is the answer.

Rather than using the top 6, or 6 scores from 3-8 or whatever, surely it would be more accurate to just use the average of all your scores. Of course, then if you only did one event and it happened to be anomalously super then you'd be overranked, but the majority of orienteers make it to more than one ranking event a year.

[graeme]

surely it would be more accurate to just use the average of all your scores.

That's what the list does for its statistics, it's only the published list which uses only top-6.
And the reason for that is that BOF insisted the system should not penalise anyone for going to an event.

[AndyC]

There is validity for that - I've seen golfers deliberately play into bunkers to avoid damaging their handicap when they were well ahead in a strokeplay head to head match.
In O someone, thinking they were having a bad run, could deliberately mispunch to avoid harming their ranking.

[greywolf]

surely it would be more accurate to just use the average of all your scores.

That's what the list does for its statistics, it's only the published list which uses only top-6.
And the reason for that is that BOF insisted the system should not penalise anyone for going to an event.

more precisely, it uses the average of your non-zero scores, so that if you have 5 1200 point runs and a mispunch your average is 1200, not 1000 (if it doesn't do it this way then it can't assess the strength of the field properly, which is essential for determining how points are distributed)

and if you base the list itself on average of non-zero scores then you incentivise deliberate DNFs from those who know they've had a bad run, which not only distorts thier ranking, but might muck yours up too (when you have a storming run on a course with a dozen or so starters and you get back to find several DNFs from those who got lost, so there are less than 10 finishers and the course is unranked...)

would this happen? I don't know - it seems a bit petty but then I've heard people complain about the SI beep because it gives away the location of the control

[King Penguin]

Years ago, white water racing rankings used to be based on averages, and there were indeed instances of people deliberately stopping and getting out before the finish line when they knew they had had a bad run, so as not to reduce their average.
I am opposed to the use of averages for published raking lists for this reason.

[m4rk]

I've heard people complain about the SI beep because it gives away the location of the control

I went to an event in Sweden where they had turned the beep off to make it harder (totally pointless; at Idrefjall it was near impossible anyway).

and there were indeed instances of people deliberately stopping and getting out before the finish line when they knew they had had a bad run, so as not to reduce their average.

Presumably, though, this could happen at the moment? Since if you mp your score isn't included in the calculation of everyone else scores, hence you avoid bringing up the average (and standard deviation) if your score is much higher than theirs.

And I think you overestimate the issue anyway; I'd say this was akin to quick M21s running say, short green courses against the W60s just to get potentially massive scores, but it doesn't seem to happen that much.