Results and Rankings.

[Spookster]

One question: when any amendments are made to the calculation method, is the new way of calculating applied to historic results?

When the system was launched, and then some initial anomalies were corrected, the points for all previous events were re-calculated. However, I don't expect that when further improvements are made (this autumn, as noted by DJM above) there will be any historic events re-calculated. The number of events is now large, and it would be too big a task. The rolling 12-month nature of the list gradually eliminates the older events anyway.

[Scott]

But there still appear to be "anomalous" score cropping up now, and it would take another whole year for them to drop out.

The ranking list only updates once a week anyway. The database isn't *that* big - I would have thought the server could do a complete re-calculation from the initial data in less than 168 hours.

[Spookster]

But there still appear to be "anomalous" score cropping up now, and it would take another whole year for them to drop out.

True. A complete re-run from the beginning may be the best solution, to properly clean out the effects of the anomalous scores.

[CS99er]

This may have cropped up before (and I was losing the will to live afetr ploughing through 10000 posts on this thread), but the blindingly obvious way of avoiding a lot of this anomaly business is to stop using the "best six scores" system. As I pointed out in the June CompassSport, not only does this automatically pick up every anomaly going and is potentially quite unstable but it can also be extremely unfair because it favours people who do more events, regardless of whether they are better orienteers.

This can be easily avoided by adopting a "mean score over the last 12 months" system. Simple, dilutes the effect of anomalies, much fairer and can be used for people with fewer than 6 ranking scores (although going below 3 is probably undesirable).

I think it's a no-brainer.

[DJM]

This can be easily avoided by adopting a "mean score over the last 12 months" system. Simple, dilutes the effect of anomalies, much fairer and can be used for people with fewer than 6 ranking scores (although going below 3 is probably undesirable).

The danger with this approach, and which is one reason why we didn't adopt it, is that those having a bad run will be tempted to mispunch, or otherwise disqualify themselves, in order not to bring their average down. Those who are semi-injured/pregnant/etc would be tempted not to enter in the first place, for similar reasons.

[andypat]

Like one or two posters above I'm worried this thread sounds a bit more negative than it ought. On the whole I love the new ranking list. Its a great motivator and I like it far better than the previous age based one. I dont have a problem with the best 6 scores provided there are enough ranking events around to get them in!

[AlanB]

This may have cropped up before (and I was losing the will to live afetr ploughing through 10000 posts on this thread), but the blindingly obvious way of avoiding a lot of this anomaly business is to stop using the "best six scores" system. As I pointed out in the June CompassSport, not only does this automatically pick up every anomaly going and is potentially quite unstable but it can also be extremely unfair because it favours people who do more events, regardless of whether they are better orienteers.

This can be easily avoided by adopting a "mean score over the last 12 months" system. Simple, dilutes the effect of anomalies, much fairer and can be used for people with fewer than 6 ranking scores (although going below 3 is probably undesirable).

I think it's a no-brainer.

Hmm, not sure about using an average. If you are improving, or coming back from injury, you are possibly going to have some low, historic scores included which don't reflect your current performances. Also you are much more likely to have included in your total an extremely low score - you get completely lost, exhausted or twist an ankle - rather than inexplicably have an outstanding (and uncharacteristic) performance resulting in you, say, winning the event!

The current system does favour those that do more events but isn't that a good thing? I've gone out of my way to get rid of my 'stinkers' and the ranking system has encouraged me to go out of my way to attend more ranking events. With more events being ranked under the new 4-tier system this should, anyway, become less of an issue.

When I became interested in my ranking scores I started looking around for more ranking events to attend. This led me to consider the Lakes 5 when I had previously thought that these type of events were not for people of my standard. That is one extra entrant that event gained, 3 more ranking scores for me (I mp'd one day ) and three more visitors for the tourist industry!

What I would like to see, if the people responsible are willing to do it, is the average for each competitor published in the rankings and the ability to sort/rank on that score. That would be informative and enable obsessives like me to have a stab at working out the ranking points on a Sunday night (or mid afternoon if its an ESOC organised event ).

[AlanB]

Like one or two posters above I'm worried this thread sounds a bit more negative than it ought. On the whole I love the new ranking list. Its a great motivator and I like it far better than the previous age based one. I dont have a problem with the best 6 scores provided there are enough ranking events around to get them in!

Totally agree and credit must go to the people responsible.

[AAH]

One question: when any amendments are made to the calculation method, is the new way of calculating applied to historic results?

...I don't expect that when further improvements are made (this autumn, as noted by DJM above) there will be any historic events re-calculated. The number of events is now large, and it would be too big a task...

Not necessarily – Adventure Racer suggested a good way of ironing out the problems arising from the initial seeding some time ago (page 14, post 2) which is to run the existing data through repetitively until it converges. That sounds to me like a really good fix which would be easy to implement. (See also graeme’s post 4 on page 8 as to how the system could/should be checked by running results through algorithm again and again.)

Thus, you could feed the results for the past year in again and again, effectively simulating a century or so of the same results taking place on the same day of the year annually, look at what the ranking are at the end of it and, providing nothing unexpected happens, paste the simulated data for 2110 over the real data for 2010... I doubt it would take the person who wrote the program long to set up a test harness to feed the results repetitively into the ranking program or otherwise tweak the program to do this. It shouldn’t be too hard to write the resulting data over the current data either.

That might get rid of a lot the issues very quickly and without much cost or effort. If the scores (surprisingly) didn’t converge it would be best to know now!

The resulting code would also be useful for future experiments with the underlying algorithm. The point calculating algorithm could be edited, the above procedure repeated, and the resulting effect viewed easily without also having to work out how to reseed a new algorithm.

In terms of the ranking list as a whole, although I’m one of those who (really) doesn’t like the algorithm based on standard deviations, I generally think the idea of a single ranking list is a (really) good idea. There is no particular reason why a single ranking list shouldn’t work well or why it couldn’t be made to work with a bit of editing… the above fix would be a good start, and not difficult or expensive. It could take (many) years for the different scores for competitors who usually run different courses to converge otherwise and it would be better for the popularity of the system if this was fixed relatively soon.

Hopefully the planned fixes for later this year will sort out some of the main gremlins, but if the planned fixes don’t include some procedure to get rid of issues arising from the original seeding, something like the above procedure might not be too hard to include?

[graeme]

This can be easily avoided by adopting a "mean score over the last 12 months" system.
I think it's a no-brainer.

"mean score" is already used to calculate the points, and the anomalies keep coming.

[peawet08]

Faster? The chap in question gained 1221 points at the British Elite Sprint Champs coming 2nd in the C final running against all age groups. That would surely suggest he deserves some credit for his current ranking?

No disrespect to the gentleman mentioned by the way but there are lots of names in the rankings, most of which I have never heard of!

[peawet08]

I have no problem with whatever ranking system is used, I am just saying that I think there should be two lists.

When I look at a ranking system and see that I am 1800th in the country that's what I expect to be. I'm 39 this year and not the fittest person in the world so it's nice to see how I fair compared to others in my age category.

Coming up I'll be doing the regional event at Tockholes and I like a long run out so am doing the Black course (I like value for money) however as I'm not very good at running with any speed for long distances this will affect my score, if I entered a shorter green course would I not gain more points because I would be more competitive?

[EddieH]

I have been a little surpised at how quickly the initiallly appalingly inaccurate seed data has ironed out no doubt due to the likes of me running all sorts of courses. When it started I had absolutle no hope of getting any worthwhile points when running against the elite because they were so woefully lowly pointed (if that makes sense )

Perhaps at the shorter end there has not been enough cross competition yet as it does seem that many older competitors, possibly more the short course ones, that have inflated points.

This is often shown up when normal longer distance runners step down to green or below, (or in the shortest course urban events where the same meetings happen).

Not running thhese courses myself I am actually quite surprised at how well my points reflect my performance - I find it impossible to fool the system even in technical areas when I make mistakes. If I knock the mistakes off my time I generally see that the points I would have got are within a very narrow band. It actually makes me think that the system might provide a pretty realistic ranking.

I just feel that some technically trivial areas, and intense local knowledge, enables runners to attain high scores at a few events - and after all they only need 6.

[AlanB]

Not necessarily – Adventure Racer suggested a good way of ironing out the problems arising from the initial seeding some time ago (page 14, post 2) which is to run the existing data through repetitively until it converges. That sounds to me like a really good fix which would be easy to implement. (See also graeme’s post 4 on page 8 as to how the system could/should be checked by running results through algorithm again and again.)

I haven't quoted all your post but thanks. I'm sure I read the posts you highlight but couldn't remember where and didn't feel like going through so many pages.

In terms of the ranking list as a whole, although I’m one of those who (really) doesn’t like the algorithm based on standard deviations, I generally think the idea of a single ranking list is a (really) good idea.

Just out of interest what would you suggest as an alternative?

[AlanB]

Coming up I'll be doing the regional event at Tockholes and I like a long run out so am doing the Black course (I like value for money) however as I'm not very good at running with any speed for long distances this will affect my score, if I entered a shorter green course would I not gain more points because I would be more competitive?

As I understand it, in the Black you would be competing against better opposition who will (in theory) have higher average ranking points. Therefore, when the comparison/normalisation process takes place you should end up with similar ranking points as doing, say, the Brown. Although you say that your average speed suffers over longer distances, it also does for those who are usually faster than you.

It will be interesting to see how you go on and whether your Black score is comparable with your usual scores.

Like you, I also like value for money and I like to run for well over the hour (especially after considerable travel). Perhaps the time to consider moving up is when you no longer think you are getting sufficient orienteering per £.