Event Structure Review: Rankings List

[awk]

You'd generally expect that the winner of brown (and sorry, but I haven't got my head round the new colours yet)

No different. The changes are: Red to become a short Green; short and long versions of Red and Green available if needed; use 'light' to mean a TD4 version of a TD5 colour. Apart from the Red, none of the colours themselves changed.

[Ed]

Out of interest, the Germans have a single ranking list system. The calculations are described (in German) in this document.

Whether the rankings are realistic and really comparable I don't know. The top of the 21s both look about right, and the leading W21 would be placed 13th on the M21 (with 87.66 points compared to 102.17 points for the leading M21). The leading M35 would meanwhile be placed 5th on M21 with 94.55 points. Probably that's not too far off.

[graeme]

ESRG agreed (by majority ) that putting a mathematically detailed appendix K to the EGM would lead to confusion.

Of course, we did produce an algorithm which we believe will be stable, details of which are at

http://www.ph.ed.ac.uk/~gja/Competition ... xK_rev.doc

[jac]

ESRG agreed (by majority ) that putting a mathematically detailed appendix K to the EGM would lead to confusion.

Of course, we did produce an algorithm which we believe will be stable, details of which are at

http://www.ph.ed.ac.uk/~gja/Competition ... xK_rev.doc

Some comments on first reading:

Having one ranking list rather than separate lists for men and women is likely to increase the number of your close rivals, possibly enhancing the appeal.

Rather than just having a 'cold start' by setting up the initial ranking points estimate based on the running speed ratios, you could start from there, pretend it is the end of 2006, and run the algorithm on all the 2007 ranking events. Then take the 'end 2007' ranking points to do a 'warm start' in 2008, assuming the point scores seem plausible. You will need to test the algorithm on representative data anyway, so this procedure will help to find anomalies *and* give a more credible starting point.

Can we get some 2007 colour-coded results to work on too? This will also help with David May's problem of initial estimation of points for Short course runners - at colour-coded events, they run against ranked Long course competitors.

Running speed ratios are not consistent - at some events, the longest courses have a high percentage of easy running and / or navigation, just to get the length right. At such events, runners on longer courses would have an advantage over those on shorter courses which were full of harder running and navigation. But better events will have long courses without trivial fast sections, so the running speed ration will differ.


jac

[NickC]

As of this year,we have a single ranking list in France. I have not yet discovered how it works but it has some strange querks - if you run a shorter course than yours and win it then you can get more points than if you ran your designated course!! However, Thierry is top of the list and so it must be OK!

[graeme]

But then again Thierry prefers running shorter courses than his designated ones...

I really don't know how the algorithm will play out in terms of which course will net you most points - my best guess is that it would be the longest you can comfortably run. My uncertainty is even greater if its extended down to orange. Of course it needs testing - David Rosen has offered to run the algorithm over 2006 data.

[greywolf]

I really don't know how the algorithm will play out in terms of which
course will net you most points - my best guess is that it would be the
longest you can comfortably run. My uncertainty is even greater if its
extended down to orange. Of course it needs testing - David Rosen has
offered to run the algorithm over 2006 data.

I'd thought that only the highest TD courses at any given event would be counted - which would rule out orange...but otherwise, shouldn't, in theory at least, a great run on green be equivalent to a great run on brown? (By which I mean, 2 equally matched elites, running spot on the expected winning times on their respective courses)

In practice, because the points scored will reflect the competition, you'll probably have to win green by miles and miles to get the points you'd get from plodding round a long brown...

[awk]

In practice, because the points scored will reflect the competition, you'll probably have to win green by miles and miles to get the points you'd get from plodding round a long brown...

Some people do! e.g.
http://www.orienteering.ilkley.org/events/archiveresults/2006/penistone_results.html

[EddieH]

As can an M10 on Light Green!!! http://www.basoc.org.uk/inshriachsouth07/5LGREE.HTM

[Adventure Racer]

There's got to be something wrong with a ranking system that gives 1100 points for winning a National , 1200 for finishing second in the following day's Badge event and 1300 for winning from two other competitors in another Badge event a few weeks later.

If you won the national by a couple of seconds, with 10 other runners within 30 seconds, were only a handful of seconds behind the winner on the badge with the rest a fair way back and then slaughtered 2 other people who are right at the top of the rankings I don't see anything at all wrong with that. It's exactly the sort of points that a good ranking list should produce given those circumstances (in that there is no reward for winning or getting a top placing, simply for having a good run - other competitors are there simply in order to be able to judge how good a run you had).

[EddieH]

But Adventure Racer, the point is that those 2 that you slaughtered in the minor events might have any number of reasons~ for not trying to put in a top performance. In the majo event they were putting their heads on the line with all out effort.

* Examples that I have often used are
- trying out a training technique
- running with injury/illness
- running after mega training
- taking photos on the way round

If ranking is focused on major events then in general these will not apply.

[Simon E]

As ever, we have different sets of people wanting different things. The bog-standard orienteer (?) wants a simple list that confirms they are better then the other people they run against every week. A vociferous minority (?) want a ranking list of the best orienteers in the country and they want it to be right (whatever that is).

I will absolutely guarantee that any complex ranking calculation as being discussed that tries to rank everybody in a single list will do nothing more than generate the "wrong" answer. Godders was top of the old ranking list because he happened to score most points based on the way the list was calculated at the time. The same will be true for any change to the calculation. The proposed weighting factor for major events might change who comes top, but doesn't necessarily give the "right" answer. Yes, people might take the big events more seriously, but if you don't make it to six weighted events you won't be top no matter who you are (Thierry excepted - possibly).

Why not stick with the IOF model? The ranking list is an interesting way of comparing the incomparable, but no-one takes it very seriously. What counts is the World Cup. This avoids fudge factors by being based on finish positions only: proper race results, not some mathematical guess at who might beat who. How many elite runners tell you their best position on the IOF ranking list on their web site? I don't think I've ever seen it done, but nearly all of them proudly reel off their World Cup results.

I'd suggest we go for a ranking list with no weighting factors and treat it as a bit of fun. Then develop the various Cup competitions to determine who you think is the best orienteer, by awarding points for real race results.

[DJM]

Godders was top of the old ranking list because he happened to score most points based on the way the list was calculated at the time

If I remember rightly, this was because (a) the event in question was a night event which, because of the huge mistakes which people can make at night, should not really be part of a ranking scheme anyway, and (b) it was at a time when squad results from World Cup and WOC were not included.

Why not stick with the IOF model? The ranking list is an interesting way of comparing the incomparable, but no-one takes it very seriously.

What an utterly unsubstantiated statement! If no-one takes it very seriously, why are there some 100 World Ranking Events put on each year, each one involving an IOF levy?

What counts is the World Cup. This avoids fudge factors by being based on finish positions only: proper race results, not some mathematical guess at who might beat who.

The World Cup scoring system produces a plausible list for the best few runners but is actually a pretty terrible system for those lower down. I don't know if Simon is referring to the pre-2007 WCup system or the current one, but the old one would match UK events better so I'll use this to give you just one reason why it doesn't work lower down the lists.

The scoring system was 50 for a win, 45 for 2nd, 41 for 3rd, 38 for 4th, 36 for 5th and then 35, 34, 33, etc for the rest of the placings down to 1 point for 40th place, which looks fine at first glance. However, the big weakness is that the points gained by most of the finishers depends far more on how many turned up to the race than on how well the runner ran. An average performance at a race where there were 40 starters might award the runner 20 points, whereas the same performance at a race where there were 100 starters (distributed over a similar ability range) would probably result in zero points. However, both the official IOF and BO rankings schemes are designed to award the runner the same number of points whether there were 40 starters or 100 starters. I know which system I believe to be fairer ...

The World Cup scoring system is designed to address only those fighting for the top few places. Its main merit is that it is transparent and easy to understand. However it is entirely inappropriate as a basis for a national ranking system.

[Gross]

You wouldn't have 100 athletes starting a World Cup race under normal circumstances... against the rules.

[DJM]

Gross,

As you well know, the (Special) Rules vary from year to year and there have been mass start World Cup races in the past with large start groups. But my main point is not to do with the World Cup but to do with what might happen if we replaced the rankings system by something similar to the World Cup scoring system for UK events. It would be far easier to gain points at an event in a remote part of the country than at one where the entry was large. This is why such a scoring system is unfair.