Old hat I know, but I do wonder.
Time after time, it appears that if older competitors are going to be able to score any decent ranking points (relatively), you need to run up. You don't need to run any better when doing so, but you do need the higher scorers (a few M21Es don't go amiss!) on the same course. Run to class, and you're sure to get stuffed when it comes to points.
I thought the ranking scheme was aimed at reflecting the quality of performance whatever course you ran. It blindingly obviously doesn't, with a distinct bias against the older focused courses that makes a nonsense of scores generated.
I don't understand the maths sufficiently to explain why, but would be interested to hear from others that do.
Ranking Scheme bias?
Moderators: [nope] cartel, team nopesport
Re: Ranking Scheme bias?
Agree completely awk. All my 6 counting scores are when I ran up on the longest course available.
I don't know why, but if you're bothered, you just need to run up. I've entered Elite at JK and BOC.
Fingers crossed my M59 body will get me round...
I don't know why, but if you're bothered, you just need to run up. I've entered Elite at JK and BOC.
Fingers crossed my M59 body will get me round...
-
Homer - diehard
- Posts: 969
- Joined: Wed Apr 13, 2005 12:10 pm
- Location: Springfield
Re: Ranking Scheme bias?
My view is that the maths works in a steady state.
The pot of points gets shared between the competitors based on ability.
The results are really quite good for a system which includes urban, easy forest, hard forest events etc.
However the state is not steady, especially due to those fast improving youngsters.
An improver going from say 900 to 1100 points whilst doing green courses will suck 200 points away from those green competitors she runs against (maybe a net 3 points from each). She may suck a 100 more points from blue competitors before she progresses to brown.
The net effect in the long term is that the pot of points available to the shorter courses is lowered.
The solution is to stop coaching immediately.
The pot of points gets shared between the competitors based on ability.
The results are really quite good for a system which includes urban, easy forest, hard forest events etc.
However the state is not steady, especially due to those fast improving youngsters.
An improver going from say 900 to 1100 points whilst doing green courses will suck 200 points away from those green competitors she runs against (maybe a net 3 points from each). She may suck a 100 more points from blue competitors before she progresses to brown.
The net effect in the long term is that the pot of points available to the shorter courses is lowered.
The solution is to stop coaching immediately.
- martin
- off string
- Posts: 47
- Joined: Wed Apr 16, 2008 1:16 pm
Re: Ranking Scheme bias?
I ran my first event as a M65 (MUV urban) yesterday. I had a good clean run and I estimate that an equivalent run on MSV would have got 40-50 more points.
Last edited by King Penguin on Tue Jan 03, 2023 11:07 pm, edited 1 time in total.
curro ergo sum
-
King Penguin - addict
- Posts: 1453
- Joined: Wed Sep 21, 2005 6:56 pm
- Location: Kendal
Re: Ranking Scheme bias?
All this happened when changes were made 5/6 (?) years ago. It used to work fine before that but I guess there were some problems for younger folk I wasn't aware of. I don't mind as long as the system encourages the young ones. If it doesn't, it could do with fixing because it no longer ranks performance accurately (as outlined by others above).
- yted
- light green
- Posts: 225
- Joined: Wed May 12, 2010 1:53 pm
Re: Ranking Scheme bias?
Does this happen only at age-class events, running a lower age, or also at colour-coded? Or if someone runs an L course of the same age when they usually run S ?awk wrote:...it appears that if older competitors are going to be able to score any decent ranking points (relatively), you need to run up.
Agree that the maths works best in a steady state. But to some extent wouldn't the impact of improving juniors be offset by older competitors slowing down?
- Snail
- diehard
- Posts: 709
- Joined: Wed Nov 30, 2005 8:37 pm
Re: Ranking Scheme bias?
interestingly I think the opposite may also be true... if you are a not particularly fast/good orienteer then running down seems to give slightly better ranking points than running your age class. That makes sense to me - if technical mistakes are your downfall the longer the course then the more scope for screw ups (although its probably not linear) and if fitness is your downfall then maintaining speed round 5k may be more likely than round 7km.
To me, this isn't a failing of the ranking system its a failing of the age class system. Really people should just run the course that's suited to their ability. People's technical ability and fitness doesn't change overnight in 5yr (or 2yr stints as a youngster), and if your age was closely related to your ability then we wouldn't have a system where someone with a Birthday in Dec moves up at the same time as someone who has a birthday in January. That's probably less of an issue for M65's than it is for M14's - but we do have the bizarre situation where people born a week a apart either side of new year end up competing in different age classes against people much (relatively speaking) older/younger than them.
To me, this isn't a failing of the ranking system its a failing of the age class system. Really people should just run the course that's suited to their ability. People's technical ability and fitness doesn't change overnight in 5yr (or 2yr stints as a youngster), and if your age was closely related to your ability then we wouldn't have a system where someone with a Birthday in Dec moves up at the same time as someone who has a birthday in January. That's probably less of an issue for M65's than it is for M14's - but we do have the bizarre situation where people born a week a apart either side of new year end up competing in different age classes against people much (relatively speaking) older/younger than them.
- Atomic
- orange
- Posts: 140
- Joined: Thu Nov 25, 2021 11:56 am
Re: Ranking Scheme bias?
martin wrote:An improver going from say 900 to 1100 points whilst doing green courses will suck 200 points away from those green competitors she runs against (maybe a net 3 points from each). She may suck a 100 more points from blue competitors before she progresses to brown.
What you have described may be a reasonable approximation of the reality of how the maths translates to the real world but I don't think it actually reflects the maths of how the ranking points work.
I think the effect the OP sees is actually that the mean of the field on the shorter courses has, unsurprisingly, a lower ranking than the longer courses. Accordingly, the starting point for the calculations is lower, in order to compensate for that the fastest runners may need to run disproportionately fast (regardless of their own age), whilst if they "run up" then the higher mean of that field "lifts" their score unless they run disproportionately slowly.
Its also worth bearing in mind that no points are available to:
1. Anyone who is not a BOF member - so quite a lot of those "improvers" (certainly in Scotland) will not actually be getting any points because they don't need to join BOF until they start entering National level events.
2. Anyone under 16 - a lot of serious orienteers have honed their TD5 skills by 16 and will outrun vets on speed, so I'm not sure too many vets are being "dragged down" the rankings by improvers leaping ahead.
and that:
3. for people gaining their first six ranking points (presumably those most likely to have a meteoric rise in points) their personal mean (used to calculate the overall mean of the field on that course) is more influenced by their recent performances than those who have the best 6 of the last 12 months.
4. anyone who's ranking points suggest they ran exceptionally well or badly compared to the rest of the field are excluded from the overall calculations (so an M21 elite entering a short green course for training reasons probably doesn't inflate the overall pool, and an W75 entering a black course to prove they still can probably doesn't deflate the mean of the scores).
The net effect in the long term is that the pot of points available to the shorter courses is lowered.
I don't think this is right. However if lots of good people run up - then it does reduce the mean of their "correct" age class which will result in those who decide not to run up getting fewer points than if everyone had run their correct class.
The solution is to stop coaching immediately.
I appreciate it was meant in jest (and it did make me smile), but actually, a lot of older competitors could consider getting some coaching tailored at their new shorter style courses if they really want to maximise their rankings! Just because you've been orienteering for 30+ years doesn't mean you have nothing to learn. Of course lots of older competitors are also quite happy just to get out and have some fun and don't really care if their route choice adds 3 minutes dragging down the rankings either.
- Atomic
- orange
- Posts: 140
- Joined: Thu Nov 25, 2021 11:56 am
Re: Ranking Scheme bias?
Atomic wrote:The net effect in the long term is that the pot of points available to the shorter courses is lowered.
I don't think this is right. However if lots of good people run up - then it does reduce the mean of their "correct" age class which will result in those who decide not to run up getting fewer points than if everyone had run their correct class
Actually, I don't think that's true either, as long as everyone (but for the odd exception) runs to form. If good people run up then the mean score of the correct age class is reduced, but also the mean standard of the runners is reduced and the two cancel out.
I did a trial calculation for 5 runners with 800, 900, 1000, 1100 and 1200 points running times of 120, 110, 100, 90 and 80 minutes, and each would have been awarded the same as their current score. If any of those runners is excluded, it has no effect at all on the scores of the others. (I know that a course with only 5 runners would have been excluded, but this was just to keep it simple.)
- roadrunner
- addict
- Posts: 1059
- Joined: Fri Sep 02, 2005 8:30 pm
Re: Ranking Scheme bias?
I don't get why anyone even cares about this. It's pretty meaningless to most orienteers most of the time.
The only time it has a minor impact is when we use if for ranking at British Championships/JK and there are procedures to take into account if someone has an artificially low ranking (e.g. long term injury).
The ranking scheme *may* be biased? So what?
The biggest issue with all this nonsense is that we the sport puts an overwhelming and unjust weight on *age classes*
The only time age classes should be relevant is at the aforementioned British Championships and JK.
The rest of the time people should be running the course that they are physically capable of doing. That means some M/W21s should be running short greens and far more M/W60 should be running Brown and Black (fell racing proves this).
Get rid of all the ridiculous league requirements that say you must run a course just because you are a certain age. That's the distorting factor.
Then you'd be able to judge if the ranking scheme had bias. Even if totally unimportant.
The only time it has a minor impact is when we use if for ranking at British Championships/JK and there are procedures to take into account if someone has an artificially low ranking (e.g. long term injury).
The ranking scheme *may* be biased? So what?
The biggest issue with all this nonsense is that we the sport puts an overwhelming and unjust weight on *age classes*
The only time age classes should be relevant is at the aforementioned British Championships and JK.
The rest of the time people should be running the course that they are physically capable of doing. That means some M/W21s should be running short greens and far more M/W60 should be running Brown and Black (fell racing proves this).
Get rid of all the ridiculous league requirements that say you must run a course just because you are a certain age. That's the distorting factor.
Then you'd be able to judge if the ranking scheme had bias. Even if totally unimportant.
- rf_fozzy
- light green
- Posts: 239
- Joined: Wed Feb 12, 2014 11:13 am
Re: Ranking Scheme bias?
martin wrote:My view is that the maths works in a steady state.
The pot of points gets shared between the competitors based on ability.
The results are really quite good for a system which includes urban, easy forest, hard forest events etc.
However the state is not steady, especially due to those fast improving youngsters.
An improver going from say 900 to 1100 points whilst doing green courses will suck 200 points away from those green competitors she runs against (maybe a net 3 points from each). She may suck a 100 more points from blue competitors before she progresses to brown.
The net effect in the long term is that the pot of points available to the shorter courses is lowered.
The system takes this into account - the amount of points to be shared out is not a static, fixed pot, but is based on the current form of those who run on the day. Thus the improving runner going from 900 to 1100 points will contribute more to the base points for a course that they run as they improve - just as a declining veteran running the same course will contribute less. Of course if that improvement is VERY rapid then there will be some delay for the system to pick this up.
- pete.owens
- diehard
- Posts: 752
- Joined: Tue Jan 15, 2008 12:25 am
Re: Ranking Scheme bias?
Atomic wrote:martin wrote:The net effect in the long term is that the pot of points available to the shorter courses is lowered.
I don't think this is right. However if lots of good people run up - then it does reduce the mean of their "correct" age class which will result in those who decide not to run up getting fewer points than if everyone had run their correct class.
1st there is no "correct" class - the system just ranks a course based on the rankings of those who happen to run it.
So if at an event a black course is added late in the day and the fastest half of the entries for the brown course decide to change then it is true that the base score for the brown course will indeed be lower. But the time needed to achieve that base score will increase since it is based on the times of those slower runners. These two effects will cancel out.
- pete.owens
- diehard
- Posts: 752
- Joined: Tue Jan 15, 2008 12:25 am
Re: Ranking Scheme bias?
King Penguin wrote:I ran my first event as a M65 (MUV urban) yesterday. I had a good clean run and I estimate that an equivalent run on MSV would have got 40-50 more points.
Yes, but presumably you won that course by a significant margin - hence your score is statistical outlier. It is a second order effect; it is not that the base score for shorter courses is systematically lower (or higher), but that the number of extra points you get for each second faster that the base score (calculated from the distribution of runners times) will disproportionately effect those who are much faster or slower than the mean. I suspect the same is true for other posters claiming a bias against shorter courses.
I happened to run on the course where the highest ever number of ranking points was awarded to the winner (due to a statistical anomaly caused by a single runner). My score for that course (from the other end of the results list) was exceptionally low.
- pete.owens
- diehard
- Posts: 752
- Joined: Tue Jan 15, 2008 12:25 am
Re: Ranking Scheme bias?
Funnily enough I just came on nopesport to see if this had been discussed. I only recently discovered it myself when I was asked to run up at the Compass Sport Cup final. I had an average run and scored the best ranking points I have had in years. Looking at minutes/km required to score points it is clearly much easier to score points on the longer courses. I mentioned it on facebook and many people responded along the lines that this has been known for years. I was very confused as in principle it seems the system should not allow this, however I have been investigating over the last few weeks and I now believe the crux of the issue is that the mathematics assumes that times in events follow a normal distribution whereas in fact they are usually skewed with the peak closer the winners time and a tail to slower times. This has the effect that the mean competitors time is not the peak time, and the tail to shorter times is narrower than that to longer times. I am working on a better formula that avoids these issues. I ran a test of it on last year's JK results and the mean ranking across all courses was stable with my scheme, whereas with the present scheme it always pushes the rankings higher. I need to work on it some more before I "go public".
By the way I also got some people ask "why do you care?" on facebook. I care because, if the scheme works correctly it should be a far better measure of how well you did than just seeing if you did well against Joe Bloggs, or whatever. Also it is used to calculate club champions, handicapping, team selection etc. so if it is biased it has a knock on effect.
By the way I also got some people ask "why do you care?" on facebook. I care because, if the scheme works correctly it should be a far better measure of how well you did than just seeing if you did well against Joe Bloggs, or whatever. Also it is used to calculate club champions, handicapping, team selection etc. so if it is biased it has a knock on effect.
- srocmapper
- string
- Posts: 15
- Joined: Tue May 28, 2019 3:21 pm
Re: Ranking Scheme bias?
Snail wrote:Does this happen only at age-class events, running a lower age, or also at colour-coded? Or if someone runs an L course of the same age when they usually run S ?
It happens at both: at age-class events, the scores tend to be lower relatively for older classes (I know it should be, but what I mean is that it's harder to score well at them); if other younger age classes are running the same course, the scores are higher than if separated out (so when an M60, I really wanted M55, M50 etc on the same course; unfortunately M65 seems to be a point where we shift down a course, and aren't matched with younger classes), whilst at colour-coded events, it becomes more obvious: run up amongst the younger faster runners, and it's easier to score more.
Several people have talked about the length of the course. I don't think that's relevant (but may be wrong). What's relevant is who is running the course: get higher scorers running (especially some decent M/W21Es!), and those of others will go up too (because I'm better at shorter high-intensity races, most of my high scores come from those anyway).
pete.owens wrote:Yes, but presumably you won that course by a significant margin - hence your score is statistical outlier. It is a second order effect; it is not that the base score for shorter courses is systematically lower (or higher), but that the number of extra points you get for each second faster that the base score (calculated from the distribution of runners times) will disproportionately effect those who are much faster or slower than the mean.
I was one of those trailing King Penguin, also in my first MUV race. Decent enough run, in amongst the runners I would expect to be, yet scoring within 1 point of my worst score in 18 months. I I don't think it was just KP who was suffering some sort of disproportional effect. I was warned beforehand this would happen!
-
awk - god
- Posts: 3224
- Joined: Sun Mar 21, 2004 5:29 pm
- Location: Bradford
Who is online
Users browsing this forum: No registered users and 36 guests