Ranking List

[madmike]

i stand corrected - sorry

[graeme]

The ranking points have all changed again.

Have they? It doesn't look like they reran the lists, just did the weely rescale which was always meant to be there.
The JK from last year just fell off the lists. Because of the deflation of the scores, it was massively overrated, and because it was big, most people were scoring from it. If they're just rescaling, then to compensate for all those bloated JK2012 score, there would be a huge readjustment.

Anyway, assuming that's how they have restarted the list, it turns out that if you're interested in ranking points now is your best ever chance to get high points for no particularly good reason (unlike this year's JK, which will be the lowest ranking-ebb ever).

[m4rk]

Given that they rescaled last week and have done so again this week, it could be that they will be doing so every week - this will correct the downward drift. But we'll have to wait 12 months for this to have worked, unless they go back and retrospectively rescale all the old scores week by week.

[King Penguin]

A statement from someone in the know would be helpful.
Ollie kindly reloaded all his data after the first rebase but if there has been another full rework and this is to be repeated time and time again then the usefulness of Ollie's stats will be diminished - not Ollie's fault.
If a rework is to be applied every week that must surely indicate a fundamental flaw in the implementation, if not the logic.
The BOF site does say "British Orienteering are currently experiencing technical difficulties..." but this is with telephone lines and not the ranking system.

[awk]

Given that they rescaled last week and have done so again this week, it could be that they will be doing so every week - this will correct the downward drift.

Like Paulo, I don't see how this can be happening. It seems that all the scores from last April have remained the same, but, as he said, all scores since have simply had 29 points added to last week's figures. Thus there has been no adjustment of last May's scores against those from the last couple of weeks: no adjustment for downward drift at all.

Again, like Paulo, I can't see the logic behind what is being done, and await an explanation with interest.

[DJM]

Apologies for the delay in replying. I wanted to make sure I had agreement for quoting Mike Hamilton first.

For the record, I asked the following question last week:

Now I know that you recently announced that "On Wednesday 3 April 2013 the ranking system will be updated. This will entail a complete re-run of the results data to normalise the ranking points awarded. There will also be a change to ensure the normalisation is an on-going process" and that the changes have now been made and the new list is on display. What would help me would be if you could detail exactly what changes have been made and, in particular, how Appendix K should be modified to reflect these.

As an aside, I had anticipated that the re-run of results data would involve a week by week rebasing of all historic data, but it seems that the week by week aspect starts now instead. The effect of this is that the dreaded points drift is still preserved in today's list and that it will take some years to disappear completely ... or have I misunderstood what's happened?

Mike's reply was:

The normalisation process as described in Appendix K is now being implemented and it may not require any changes at this time.
Running a week by week historical normalisation process on results already in the system was never an option. To do so would have meant setting up a workstation, exporting all the results data and generating a data file that could be exported for import back into the web based system. That would have involved considerable developer time and was not cost effective.
The developers have normalised the data in the system as a totality through to the day of the news item and have modified the code so that the normalisation process as described in Appendix K is now implemented weekly against the data held for the previous 13 months.
The points drift has been normalised and addressed for the entire data set - the amount of drift should now be minimal and what there is should be picked up by the weekly normalisation.
None of this normalisation will take into account an improvement of running speed say during the last year as compared to 4 years ago as running speed is not a factor of the algorithm except as far as the original seed data is concerned.

[HarryO]

So - for the layman - does this explain why the Ranking Points have changed again? And does this mean that they going to change EVERY WEEK?!

[graeme]

Yes and no.
The actual numbers will change every week, but the scaling won't affect your position in the ranking list*. Of course your position will change as new results come in for you and others, and as old results drop off.

The disappearance of the JK2012 results is likely to be the biggest change we'll ever see. I think the points drift problem will take a year to clear away*. The old-man bias from the seed data will probably last for longer.


* Except in certain very weird situations to do with rounding and how widely spread your scores are.

[King Penguin]

Bit of a downer for Ollie's excellent service if points from events in the past will continue to change every week. I am struggling to understand why this should be appropriate, even if relative positions remain unchanged. Each event is run once, runners have just one result so why should points be recalculated for the next 52 weeks ?

[graeme]

They have to rescale because the algorithm is unstable due to skew. And there's no sensible way to rescale without rescaling everything.
By using a system with negligible drift they could have done better. But they decided not to.

[mike g]

How hard would it be to change the algorithm so that points don't drift? That has to be a better solution than changing everyone's points for old events every week.

I also still don't understand how the points changes have done anything to cure the problem, if a score from 11 months ago has been increased by the same amount as a score from last week.

[Big Jon]

Results in, garbage out!

[HarryO]

I know its been said before, but I strongly believe that ranking points should only be calculated for TD5 courses.
I have a friend who only runs Orange, but regularly gets more points than me

[Spookster]

I have a friend who only runs Orange, but regularly gets more points than me

Really? Which events have had an Orange course at which runners scored lots of ranking points?

[DavidJ]

I have a friend who only runs Orange, but regularly gets more points than me

Really? Which events have had an Orange course at which runners scored lots of ranking points?

He didn't actually say they "scored lots of ranking points", just "more points than me". They are not the same thing!