I'm starting a new thread on a specific issue re new ranking algorithm.
As an Ultra Vet, the new system is having an adverse effect on the ability to score any ranking points at events, when previously they would have been awarded.
2 examples.
M.V. Epsom event. 14 eligible scorers in Course 5, Womens UV, HV, Mens HV. No points awarded.
London Evening Sprint, Wapping. 10 scorers Course 5, no points awarded.
I don't fully understand all the complicated workings of how points are scored, but the reality is that a major incentive for taking part in these events is apparently being removed under this system, for reasons which aren't obvious to me. At least with the previous system we knew where we stood i.e. 10 scorers meant a result.
Any comments/explanations? It feels to me as if I'm being discriminated against for being over 65!
BOF Updated Ranking Algorithm - effect on older vets.
Moderators: [nope] cartel, team nopesport
Re: BOF Updated Ranking Algorithm - effect on older vets.
xxx wrote:I'm starting a new thread on a specific issue re new ranking algorithm.
As an Ultra Vet, the new system is having an adverse effect on the ability to score any ranking points at events, when previously they would have been awarded.
2 examples.
M.V. Epsom event. 14 eligible scorers in Course 5, Womens UV, HV, Mens HV. No points awarded.
London Evening Sprint, Wapping. 10 scorers Course 5, no points awarded.
I don't fully understand all the complicated workings of how points are scored, but the reality is that a major incentive for taking part in these events is apparently being removed under this system, for reasons which aren't obvious to me. At least with the previous system we knew where we stood i.e. 10 scorers meant a result.
Any comments/explanations? It feels to me as if I'm being discriminated against for being over 65!
Agreed!
- denbydale
- green
- Posts: 337
- Joined: Tue Apr 10, 2012 1:42 pm
Re: BOF Updated Ranking Algorithm - effect on older vets.
It's not just Older Vets. There would appear to be a number of courses that have similar discrepancies. It is "interesting" that the 2 courses mentioned do have points calculations listed but no points allocated. A not so quick scan of the potential points that could be allocated does not, on the face of it, suggest that there are any hugely anomolous scores.
It would take a bigger brain than mine to give a definitive explanation, possibly one that belongs to a physics professor
It would take a bigger brain than mine to give a definitive explanation, possibly one that belongs to a physics professor
- MIE
- green
- Posts: 370
- Joined: Fri Jan 11, 2008 10:05 pm
Re: BOF Updated Ranking Algorithm - effect on older vets.
The system is based on statistics, and it also requires to run automatically. Complaints about previous systems have centred on dodgy scores. So the algorithm tries to detect when the statistics are likely to be dodgy. Nothing discriminates against over 65s: all senior age groups are treated equally.
What has changed is that in calculating scores, we now exclude people with "extreme" scores - which is taken as more than 100 (potential) points different from the average. This means that if some elite runs on granny's entry, or someone misses their timed start, or stops for a long time to help an injured competitor before completing their run, the rankings list automatically detects something odd and the scores don't get messed up.
You will still a super good score from a super good run, provided the scores of the other competitors make sense.
To compensate from removing extra people from the statistics, we lowered the number of non-anomalous scores required to 8.
Taking the example of the MV Epsom event, there isn't enough data online to easily analyse which of the 15 runners are excluded, but immediately I see a very atypical result for Dave Ryder and Dianne O'Donohue has no previous scores: you can dig further if you like.
The course winner would get 1130 (her previous 20 score ranges from 796 to 1027). That's a massive difference, a bigger anomaly than the so-called greatest result of a generation and suggests that in this case the algorithm has done a good job identifying that the statistics from this course are not very reliable.
Disclaimer: I tested the new system, but as not thoroughly as I could have if they'd shown me the code, or even told me who the developers actually are. I one found a mistake in the implementation. There may be others. The developers group told BOF about it. They released it anyway.
What has changed is that in calculating scores, we now exclude people with "extreme" scores - which is taken as more than 100 (potential) points different from the average. This means that if some elite runs on granny's entry, or someone misses their timed start, or stops for a long time to help an injured competitor before completing their run, the rankings list automatically detects something odd and the scores don't get messed up.
You will still a super good score from a super good run, provided the scores of the other competitors make sense.
To compensate from removing extra people from the statistics, we lowered the number of non-anomalous scores required to 8.
Taking the example of the MV Epsom event, there isn't enough data online to easily analyse which of the 15 runners are excluded, but immediately I see a very atypical result for Dave Ryder and Dianne O'Donohue has no previous scores: you can dig further if you like.
A not so quick scan of the potential points that could be allocated does not, on the face of it, suggest that there are any hugely anomolous scores.
The course winner would get 1130 (her previous 20 score ranges from 796 to 1027). That's a massive difference, a bigger anomaly than the so-called greatest result of a generation and suggests that in this case the algorithm has done a good job identifying that the statistics from this course are not very reliable.
Disclaimer: I tested the new system, but as not thoroughly as I could have if they'd shown me the code, or even told me who the developers actually are. I one found a mistake in the implementation. There may be others. The developers group told BOF about it. They released it anyway.
Last edited by graeme on Thu Sep 14, 2017 12:56 pm, edited 1 time in total.
WOC2024 Edinburgh
Test races at SprintScotland (Alloa/Falkirk) and Euromeeting (near Stirling).
Test races at SprintScotland (Alloa/Falkirk) and Euromeeting (near Stirling).
-
graeme - god
- Posts: 4723
- Joined: Thu Feb 12, 2004 6:04 pm
- Location: struggling with an pɹɐɔ ʇıɯǝ
Re: BOF Updated Ranking Algorithm - effect on older vets.
graeme is spot on in his explanation - with the exception of one significant typo: instead of "we lowered the number of non-anomalous scores required to 6" he should have written 8.
I've now asked for the full data file for this course so that we can confirm (or otherwise) that the updated algorithm is doing what it should do and I'll let you know the post mortem results in due course ...
I've now asked for the full data file for this course so that we can confirm (or otherwise) that the updated algorithm is doing what it should do and I'll let you know the post mortem results in due course ...
- DJM
- diehard
- Posts: 978
- Joined: Sun Sep 30, 2007 8:19 pm
- Location: Wye Valley
Re: BOF Updated Ranking Algorithm - effect on older vets.
graeme wrote: Nothing discriminates against over 65s: all senior age groups are treated equally.
Someone hasnt heard of indirect discrimination then.....
Indirect discrimination is when there's a practice, policy or rule which applies to everyone in the same way, but it has a worse effect on some people than others.
The requirement for a set number of entrants in a course clearly does indirectly discriminate agasint anyone who routinely runs on courses which may have less than 8 (If I'm understanding the numbers) counting scorers. In this case probably M21s, definitely M/W75s and possibly one or two other groups.
My question is this
If the system can remove anomalous scores automatically - why do we need a minimum number at all? Or why cant it be say 3?
Orienteering - its no walk in the park
- andypat
- god
- Posts: 2856
- Joined: Sat Apr 11, 2009 9:58 pm
- Location: Houston, we have a problem.
Re: BOF Updated Ranking Algorithm - effect on older vets.
10 ran course 5 at Wapping last Friday - no points awarded
Last edited by denbydale on Tue Sep 12, 2017 4:23 pm, edited 1 time in total.
- denbydale
- green
- Posts: 337
- Joined: Tue Apr 10, 2012 1:42 pm
Re: BOF Updated Ranking Algorithm - effect on older vets.
andypat wrote:If the system can remove anomalous scores automatically - why do we need a minimum number at all? Or why cant it be say 3?
Presumably for the same reasons as before when it was 10 competitors, you still need a certain number to ensure the whole lot aren't anomalous.
Andrew Dalgleish (INT)
Views expressed on Nopesport are my own.
Views expressed on Nopesport are my own.
- andy
- god
- Posts: 2455
- Joined: Thu Nov 06, 2003 11:42 pm
- Location: Edinburgh
Re: BOF Updated Ranking Algorithm - effect on older vets.
andy wrote:andypat wrote:If the system can remove anomalous scores automatically - why do we need a minimum number at all? Or why cant it be say 3?
Presumably for the same reasons as before when it was 10 competitors, you still need a certain number to ensure the whole lot aren't anomalous.
Exactly so: there's a compromise between accuracy and inclusivity. We tested it against historical data and got to 8, and DJM says and I misremembered. DJM also produced a very convincing graph show in a steady decrease in "How different scores are from people's average" against "number of runners on course". Though maybe I shouldn't have needed convincing, because it is just maths...
Regarding indirect discrimination via small courses: this cuts two ways. Inevitably the scores are dodgier on small courses, and they may be dodgy-high or dodgy-low. Since your ranking is your best 6 scores (i.e. the dodgy-highs not the dodgy-lows), people running small courses can benefit from this. Older competitors also benefit indirectly from people who run down when injured.
It's difficult to quantify the benefits of running short courses without knowing why people run down. But many people believe you can get better scores running down and winning, which suggests that older competitors are over-ranked.
WOC2024 Edinburgh
Test races at SprintScotland (Alloa/Falkirk) and Euromeeting (near Stirling).
Test races at SprintScotland (Alloa/Falkirk) and Euromeeting (near Stirling).
-
graeme - god
- Posts: 4723
- Joined: Thu Feb 12, 2004 6:04 pm
- Location: struggling with an pɹɐɔ ʇıɯǝ
Re: BOF Updated Ranking Algorithm - effect on older vets.
And if I've understood it correctly, the use of a fixed 100 point difference (rather than a percentage difference) for all runners to identify anomalous results to be excluded is more likely to favour 65+s than elites, in that a wider range of "not quite anomalous enough to be excluded" results will be retained, so leading to the small course benefit that Graeme describes.
If people are that bothered about ranking points they can always enter a course with a higher expected number of entries - but the downside is that that makes the situation for those who don't even worse.
If people are that bothered about ranking points they can always enter a course with a higher expected number of entries - but the downside is that that makes the situation for those who don't even worse.
- Snail
- diehard
- Posts: 706
- Joined: Wed Nov 30, 2005 8:37 pm
Re: BOF Updated Ranking Algorithm - effect on older vets.
Based on an extemely small and not statistically acceptable sample (i.e. 1 x M55) I think I detect reduced scores from urban events and increased scores from terrain events. Before the change all my top 6 were urban / sprint; now 2 are terrain and my position has dropped. Possibly influenced by differing standard deviations and skews between the two sets ?
curro ergo sum
-
King Penguin - addict
- Posts: 1452
- Joined: Wed Sep 21, 2005 6:56 pm
- Location: notloB
Re: BOF Updated Ranking Algorithm - effect on older vets.
King Penguin wrote:Based on an extemely small and not statistically acceptable sample (i.e. 1 x M55) I think I detect reduced scores from urban events and increased scores from terrain events.
Interesting. I was hoping for a good score (for me) after winning my class (MUV) at the London City race and beating quite a number of those ahead of me in the rankings. In fact it won't even make my best 6. However, most of my top scores remain as urban/sprint events. Perhaps it is simply telling me that my other results were more impressive than I realised.
- DavidJ
- light green
- Posts: 289
- Joined: Mon Feb 18, 2008 10:37 pm
- Location: Berkshire, UK
Re: BOF Updated Ranking Algorithm - effect on older vets.
Maybe the Barbican Factor had a disruptive influence with some 'slower' people (or Barbican Veterans) doing relatively well?
- Gnitworp
- addict
- Posts: 1093
- Joined: Wed Mar 22, 2006 1:20 am
Re: BOF Updated Ranking Algorithm - effect on older vets.
Gnitworp wrote:Maybe the Barbican Factor had a disruptive influence with some 'slower' people (or Barbican Veterans) doing relatively well?
For the record, this was my first London City Race
- DavidJ
- light green
- Posts: 289
- Joined: Mon Feb 18, 2008 10:37 pm
- Location: Berkshire, UK
Re: BOF Updated Ranking Algorithm - effect on older vets.
Not mine, and I got two of the route choices wrong in the Barbican that you got right despite two previous runs there
- Gnitworp
- addict
- Posts: 1093
- Joined: Wed Mar 22, 2006 1:20 am
Who is online
Users browsing this forum: No registered users and 90 guests