Ranking Scheme bias?
Moderators: [nope] cartel, team nopesport
Re: Ranking Scheme bias?
You're an M45. The big difference I noticed (as have many others), comes in the older classes. It's very much an issue in practice!
-
awk - god
- Posts: 3224
- Joined: Sun Mar 21, 2004 5:29 pm
- Location: Bradford
Re: Ranking Scheme bias?
awk wrote:MChub wrote:awk wrote:Exactly - it's biased against those older 'winners and losers' who run to class and towards those who don't, requiring competitors to choose between rankings and peer group competition.
People don't usually choose one or the other all the time. As long as you have enough events where your result is not anomalously high compared to the rest, you overall score should be OK.
You're right, there are plenty of other criteria upon which we choose which course to run, but if you want to maximise your ranking score and want to run against your peer group, then as an older competitor you definitely have to choose, because the two are pretty much incompatible. And that shouldn't be the case. I certainly don't regard my score as 'OK' when I know that just by running out of class I would have almost automatically received a better score.
Correct me if I'm wrong, but it sounds like what you want to know is your run on course y better than your mate's run on course z.
Or that your run was better than the winner on the longest course or similar.
The rankings do not tell you that. And I suspect that any sane system can never tell you that.
Just like comparing mins/km on two different courses is broadly misleading. Just because you ran 5mins/km for 5.4km, it doesn't mean you'd run 5mins/km for 11km.
Similarly, it doesn't follow if you win your course by 23mins, then the score that you got from comparing your performance on that course on that day to the other competitors on that course on that day is comparable to a score that would be achieved if you'd run another course on the same day and had a different result. Which is what the ranking system does. More or less.
Btw, I still think it doesn't matter. Because they're not used for anything except for the occasional seeding for championships races and we can manually take into account things like returning from injury.
You might argue that your mate <insert name here> who is also an <insert category here> and decently competitive, regularly runs Black courses and so gets, on average, more ranking points than you (on average)* and so therefore when it comes to a championship, they might get a "better" start time than you. Ok. That I concede might be true. But it probably doesn't mean a huge amount if you start 30mins before the last start and they start 10mins before the last start. Either of you could still win if other things are equal. One might argue if they're regularly running black courses and you blue, then the fitness quotient might put them favourite.
I remember doing my controller grade C course and one of the questions we were asked was whether an orienteering course (as long as it meets the specs) had to be more focussed on running, navigation or both. The answer was it doesn't matter - that's up to the planner.
This is roundabout way of saying is that it feels like what you want to say (and again I may be misreading) is that <person A> may be faster, but you're the better **technical** orienteer. When in reality both are important.
I say this as someone who is currently terminally injured and may never be able to run properly again. Just because I've picked up a few 'technical skills,' if I "run" a course in 30mins and do 4.3km to complete, it doesn't make me a better orienteer than someone who runs the same course on the same day and runs all over doing 7km but does it in 23mins.
*Actually running Black courses is a bad example as too few people (because of the artificial age categories we load onto events) often run them - there aren't really enough MO (for several reasons) and people willing to run up, so usually it can be a struggle to get 10 people with results and rankings. So the distributions can be quite skewed.
- rf_fozzy
- light green
- Posts: 239
- Joined: Wed Feb 12, 2014 11:13 am
Re: Ranking Scheme bias?
MChub wrote:Ultimately, what matters is: are there any people with enough (say, 10 or more) ranking scores per year whose ranking is obviously rubbish? I don't know any; certainly, those I know in my club, from M21 to W80, are in roughly the right order.
The top end of M65 is certainly not as it should be, or as it would be based on those people running the same course at the same event, or as S5D will show it should be. I predict between 4 and 7 of those currently ranked below me will finsish ahead of me in the overall S5D scores.
curro ergo sum
-
King Penguin - addict
- Posts: 1453
- Joined: Wed Sep 21, 2005 6:56 pm
- Location: Kendal
Re: Ranking Scheme bias?
rf_fozzy wrote:Correct me if I'm wrong, but it sounds like what you want to know is your run on course y better than your mate's run on course z. Or that your run was better than the winner on the longest course or similar. The rankings do not tell you that. And I suspect that any sane system can never tell you that. Just like comparing mins/km on two different courses is broadly misleading. Just because you ran 5mins/km for 5.4km, it doesn't mean you'd run 5mins/km for 11km.
I agree you can't compare mins/km - but the ranking scheme says that it can compare performances. My X points from the Y course is used to compare directly with my mate's A points on the B course. If it can't do that (and I agree it's not doing that at present), then the ranking scheme should not be presented in the way it does, where it does exactly that.
Btw, I still think it doesn't matter. Because they're not used for anything except for the occasional seeding for championships races and we can manually take into account things like returning from injury.
At the end of the day, no leisure competition 'matters'. Especially if you're not interested or not taking part. Just as orienteering doesn't 'matter' to the majority of the population, or the finer points of urban maps and specs don't ultimately 'matter'. It might not matter to you, it does matter to others.
This is roundabout way of saying is that it feels like what you want to say (and again I may be misreading) is that <person A> may be faster, but you're the better **technical** orienteer. When in reality both are important.
Yes, you have misread my arguments. I'm simply saying that if the national ranking scheme is to function as it claims to, that X points on one course should mean the same (and be as hard/easy to achieve) as they do on any other course. If they don't, then it's not fit for the purpose it's set up for. And it's clear that they don't. (BTW British Orienteering openly says that using someon'es 6 best scores is not as reliable as using their average - which makes sense - so why are the 6 best scores used? Isn't that just comparing extremes/anomalies, and emphasising any inaccuracies in individual scores?).
.'if I "run" a course in 30mins and do 4.3km to complete, it doesn't make me a better orienteer than someone who runs the same course on the same day and runs all over doing 7km but does it in 23mins.
Agreed - it might demonstrate you're a better navigator/technician, but orienteering is, as you say, the combination between running and navigation. I''ve never argued anything different, rather the opposite.
-
awk - god
- Posts: 3224
- Joined: Sun Mar 21, 2004 5:29 pm
- Location: Bradford
Re: Ranking Scheme bias?
awk wrote:BTW British Orienteering openly says that using someon'es 6 best scores is not as reliable as using their average - which makes sense - so why are the 6 best scores used? Isn't that just comparing extremes/anomalies, and emphasising any inaccuracies in individual scores?
The general principle is that you shouldn't be "punished" (in terms of your ranking) for going orienteering rather than sitting at home. Ranking people by their average scores would mean, for example, that people who get injured and go orienteering before they are back to full fitness would see their position in the ranking list depressed for a year until the lower scores age off, even if they go back to winning everything once they are back to full fitness. It would also invite such silliness as people who feel that they've had a bad run (and who care about where they appear in the ranking list) deliberately mispunching on the last control to avoid slipping down the rankings as a result.
That said, the average is more reliable, which is why it is used in the ranking calculations - just not in the way that the rankings are then presented.
British Orienteering Director | Opinions expressed on here are entirely my own, and do not represent the views of British Orienteering.
"If only you were younger and better..."
"If only you were younger and better..."
-
Scott - god
- Posts: 2384
- Joined: Wed Jan 17, 2007 4:43 am
- Location: in the queue for the ice-cream van
Re: Ranking Scheme bias?
I have thought, and suggested previously elsewhere on Nopesport, that perhaps rankings should be based on each person's best 2nd-7th best results (or 2nd-6th if only using 5 to avoid needing one more event for a full set) i.e. exclude their best one. Whilst some people do get one abnormally high outlier it is much rarer for someone to have two abnormally high results.
i.e. akin to other sports scored by judges where the highest & lowest scores are discarded to try to avoid individual tactical / political voting.
i.e. akin to other sports scored by judges where the highest & lowest scores are discarded to try to avoid individual tactical / political voting.
curro ergo sum
-
King Penguin - addict
- Posts: 1453
- Joined: Wed Sep 21, 2005 6:56 pm
- Location: Kendal
Re: Ranking Scheme bias?
Fair points Scott, although as a result we are emphasising the outlying runs - the anomalies - which others here tell me are not to be relied on!
That makes more sense John, although some of us have actually managed 2 anomalous results! Many of the arguments here have talked of outlying results being unreliable, needing to be discounted etc, yet our ranking scores are based on them. I would also expect a national ranking scheme to require a minimum number of higher grade events to be included.
That makes more sense John, although some of us have actually managed 2 anomalous results! Many of the arguments here have talked of outlying results being unreliable, needing to be discounted etc, yet our ranking scores are based on them. I would also expect a national ranking scheme to require a minimum number of higher grade events to be included.
-
awk - god
- Posts: 3224
- Joined: Sun Mar 21, 2004 5:29 pm
- Location: Bradford
Re: Ranking Scheme bias?
Last weekend I got my best ranking points since 14/9/2019. How ? By entering as if I was 3 decades younger than I am.
The winner on the designated course for my age class got 100 points less.
Last weekend was a terrain event. The better score 4 years ago was in a sprint, where I "always" score higher. I have to go back to 30/6/2016 (over 7 years) to find higher points from a terrain event.
One result is hardly statistically valid, but it is further evidence to support the other comments in this thread.
The winner on the designated course for my age class got 100 points less.
Last weekend was a terrain event. The better score 4 years ago was in a sprint, where I "always" score higher. I have to go back to 30/6/2016 (over 7 years) to find higher points from a terrain event.
One result is hardly statistically valid, but it is further evidence to support the other comments in this thread.
curro ergo sum
-
King Penguin - addict
- Posts: 1453
- Joined: Wed Sep 21, 2005 6:56 pm
- Location: Kendal
Re: Ranking Scheme bias?
King Penguin - did you look and see how many other people your age were running up?
Say you are an M65, but ran in M50. Were there others that did this too? or in M45/55/60 etc? It may not be the rankings that are broken but rather the age classes. If lots of people are regularly running outside their age it is probably a sign it is a naff way to lump people together. I assume people mostly run outside their age because they want to run a different (longer/shorter) course rather than in some contrived way to get a better ranking?
Say you are an M65, but ran in M50. Were there others that did this too? or in M45/55/60 etc? It may not be the rankings that are broken but rather the age classes. If lots of people are regularly running outside their age it is probably a sign it is a naff way to lump people together. I assume people mostly run outside their age because they want to run a different (longer/shorter) course rather than in some contrived way to get a better ranking?
- Atomic
- orange
- Posts: 140
- Joined: Thu Nov 25, 2021 11:56 am
Re: Ranking Scheme bias?
Well, well!
I was that winner on the designated course, Short Blue, where I scored 1168 with 42:49 for 3.6k.
Running the same speed on Blue, a very similar course just 300m longer, I would have scored 1220 pts. Or, to put it another way, I could have run 25% slower to score the same points.
It's farcical. And it's not an isolated case. So much so, I knew beforehand that running to class would cost me (a lot of) ranking points, however well I ran.
No, it's not the age classes that are broken - it's the rankings. For the rankings to work in the way they are presented, they have to work across courses/classes, where X points on one course/class is worth the same as X points on another course/class. They clearly don't as illustrated above (and time and time again ad nauseaum). That's completely age independent. It shouldn't matter who else is on your course, but it clearly does. I can, and regularly do, score higher points for a mediocre run on a course populated by high ranked runners than I ever have for an excellent run on a course populated by lower ranked. It's been even more glaringly obvious since moving into M65, where my only decent scores have been running up, even though most of them have been fairly mediocre runs. It also means for older runners that it's much easier to score points at an obscure regional event with a few M/21Es running the same course than with all the best in your age class assembled for a championship.
Basically, I would suggest that the ranking list probably works fine for anybody running the 'higher' courses. The further away from that you get, whether it's older classes at championships etc, or shorter colours at regional and other events, the harder it gets to score points. That's unless those faster runners run down (and they don't).
I was that winner on the designated course, Short Blue, where I scored 1168 with 42:49 for 3.6k.
Running the same speed on Blue, a very similar course just 300m longer, I would have scored 1220 pts. Or, to put it another way, I could have run 25% slower to score the same points.
It's farcical. And it's not an isolated case. So much so, I knew beforehand that running to class would cost me (a lot of) ranking points, however well I ran.
Atomic wrote:Say you are an M65, but ran in M50. Were there others that did this too? or in M45/55/60 etc? It may not be the rankings that are broken but rather the age classes. If lots of people are regularly running outside their age it is probably a sign it is a naff way to lump people together. I assume people mostly run outside their age because they want to run a different (longer/shorter) course rather than in some contrived way to get a better ranking?
No, it's not the age classes that are broken - it's the rankings. For the rankings to work in the way they are presented, they have to work across courses/classes, where X points on one course/class is worth the same as X points on another course/class. They clearly don't as illustrated above (and time and time again ad nauseaum). That's completely age independent. It shouldn't matter who else is on your course, but it clearly does. I can, and regularly do, score higher points for a mediocre run on a course populated by high ranked runners than I ever have for an excellent run on a course populated by lower ranked. It's been even more glaringly obvious since moving into M65, where my only decent scores have been running up, even though most of them have been fairly mediocre runs. It also means for older runners that it's much easier to score points at an obscure regional event with a few M/21Es running the same course than with all the best in your age class assembled for a championship.
Basically, I would suggest that the ranking list probably works fine for anybody running the 'higher' courses. The further away from that you get, whether it's older classes at championships etc, or shorter colours at regional and other events, the harder it gets to score points. That's unless those faster runners run down (and they don't).
-
awk - god
- Posts: 3224
- Joined: Sun Mar 21, 2004 5:29 pm
- Location: Bradford
Re: Ranking Scheme bias?
Atomic wrote:King Penguin - did you look and see how many other people your age were running up?
I am M65: I entered Black which was designated for M21 only.
Running on that course were :
1 x M20
13 x M21
1 x M35
1 x M40
4 x M45
1 x M55
1 x M65
curro ergo sum
-
King Penguin - addict
- Posts: 1453
- Joined: Wed Sep 21, 2005 6:56 pm
- Location: Kendal
Re: Ranking Scheme bias?
King Penguin wrote:
I am M65: I entered Black which was designated for M21 only.
*rolls eyes*
And this obsession with age classes when it is not at all in the slightest bit relevant to the competition is what is wrong with orienteering at the moment.
Black is not designated for M21 only.
You can and should be able to choose any course you want to do at any event, except for championships. This is the problem with too numerous tin pot leagues.
We have to stop this idea that Black = M21, Green = W55 or whatever. People should do whatever course they want. No matter their age.
This would then sort the irrelevant complaining about "my course doesn't give me enough points" too.
- rf_fozzy
- light green
- Posts: 239
- Joined: Wed Feb 12, 2014 11:13 am
Re: Ranking Scheme bias?
I do agree that those leagues that only give full points/scores if people run specific colours are a terrible break on participation. Can we put ourselves in the shoes of a 21 year old, perhaps towards the healthy end of the general population but not like the super athletes who are competitive on the longest courses, that they must run black (or not qualify for the competition)? Even worse for older juniors where for example 14 year old boys get told they should run blue, and 16 year olds short brown. No wonder the drop out rate at these ages is so high.
What does it take to change the rules on these leagues, or are we content to let the sport slowly fade into something done occasionally on the odd winter weekend and summer holidays in Scotland?
What does it take to change the rules on these leagues, or are we content to let the sport slowly fade into something done occasionally on the odd winter weekend and summer holidays in Scotland?
- SeanC
- god
- Posts: 2251
- Joined: Wed Mar 16, 2005 6:46 pm
- Location: Kent
Re: Ranking Scheme bias?
rf_fozzy wrote:And this obsession with age classes when it is not at all in the slightest bit relevant to the competition is what is wrong with orienteering at the moment.
*rolls eyes*
The age classes are irrelevant. In fact, King Penguin's answer underlines how irrelevant they are - showing the range of people running the Black.
The problem is not age classes, the problem is the ranking scheme. X points should represent the same whether scored on the Black, the Green or whatever course - the ranking scheme depends on that being the case to function in the way it is presented. What is transparently clear is that it doesn't - X points are easier to score on the courses where there are higher ranked people. If all of them ran the Green, and all the lowest ranked ran the Black, then it would be easier to score on the Green.
If someone chooses to compete in a 'tin pot' league or not, it shouldn't make a jot of difference - they should find it just as easy/hard to score a certain amount of points on whatever course they run. The fact that they don't is nothing to do with the age class competition, but everything to do with the failings of the ranking scheme.
Having said that, I do agree with SeanC that always putting the M21 competition (or older juniors) on the longest, hardest course, with no real alternative, is self-destructive. But that's not relevant to the issue with the ranking scheme.
-
awk - god
- Posts: 3224
- Joined: Sun Mar 21, 2004 5:29 pm
- Location: Bradford
Re: Ranking Scheme bias?
awk wrote:
The age classes are irrelevant.
Then simply just run the longer courses if the ranking points are, for some unknown reason, important to you.
- rf_fozzy
- light green
- Posts: 239
- Joined: Wed Feb 12, 2014 11:13 am
Who is online
Users browsing this forum: No registered users and 10 guests