Results and Rankings.

[graeme]

Too much messenger-shooting going on here methinks

comparison of mins/km ... little real justification

Every BOF guideline to planners for estimating course lengths is based around mins/km. Are you saying they're all wrong? awk has his real justification in that
these guidelines are implemented, accepted, and widely agreed to be good. No doubt you can find odd cases to the contrary (e.g sprint/urban where the km is meaningless): but the problem is not of that stochastic type, rather it is systematic overranking on shorter courses.

I would argue that, rather than seeing where your mins/km would place you on a different course, it's more reliable to see where your rankings points would put you instead!

Few people swap classes often enough to do this. As it happens, I do. And when I run M21, the ranking points I gained from M45 typically predict I should finish half a standard deviation (100points) higher than I actually do.

[EddieH]

Perhaps M45 are overranked Graeme. My 3 highest scores were all running M45

[graeme]

Nah, you just got points for beating up on the English in technical terrain. Any idiot can do that

[Miner]

I don't pretend to understand the stats, but if the recalculation is supposed to address anomalies on Light Green the points for my only (very slow) Light Green run have increased by 33 while my best points on Green have decreased by 70.

[DJM]

graeme wrote:
Every BOF guideline to planners for estimating course lengths is based around mins/km. Are you saying they're all wrong?

Of course not!! The ratios give the best estimate for selecting which length goes with which course. But they also come with a health warning as it's recognised that other factors can upset the ratios and must thus be taken into account. For example, Guideline B (Course Planning) says

2.6.3 Applying the course length ratios – points to watch for:
• M21 probably uses the whole area. The shorter courses use only part of it, and this might be more or less runnable, or steeper/flatter, than the average
• rough terrain has a greater effect on the running speed of younger and older competitors than of M21s
• older competitors are significantly affected by steep terrain, particularly downhill
• older competitors find dense tree growth more of an obstacle – suppleness decreases with age
There is no magic formula for allowing for these variables.

So, start with the published ratios and then use skill and judgement to vary them according to where the courses go and how they are planned.

[DJM]

graeme wrote:
Few people swap classes often enough to do this. As it happens, I do. And when I run M21, the ranking points I gained from M45 typically predict I should finish half a standard deviation (100points) higher than I actually do.

Is this still the case, now that the algorithm has been corrected? If so, could you send me some recent examples of this effect?

[IanD]

graeme wrote:
Few people swap classes often enough to do this. As it happens, I do. And when I run M21, the ranking points I gained from M45 typically predict I should finish half a standard deviation (100points) higher than I actually do.

I guess the reason a standard M45 is ranked lower than a standard M21 is to do with the lower running speed of the aged legs. If the standard M45 can also not maintain his normal speed for as long as the standard M21, that would not be reflected in the ranking points, because M45 courses are shorter than M21.

So if a standard M45 can only maintain his (slower) speed for the length of an M45 course, and his performance falls further should he run M21, it would be reasonable for him not to attain the position on M21 that the ranking points might have suggested.

I've no idea whether this is the case for Graeme. But it would be interesting to know whether he gets closer to the position which his ranking points suggest when running M21 in a sprint rather than a long race.

[AlanB]

DJM

Is it possible to state (briefly and for the non-statistician) how the algorithm has been changed, and what the object of the change was?

Cheers

[AAH]

It is possible to tell by inspection that the issues arising from the original seeding have not been fixed. If that issue was fixed, some people who usually run black/long etc. would suddenly gain a points and places and people who regularly run green/blue/short would suddenly lose a lot of points and places.

Tweaking the algorithm won’t fix the seeding issues (unless the system is left to run for a decade or three), the seeding needs fixed. Fixing the seeding might be easy – I’ll repeat a chunk of my post from page 16…

… Adventure Racer suggested a good way of ironing out the problems arising from the initial seeding some time ago (page 14, post 2) which is to run the existing data through repetitively until it converges. That sounds to me like a really good fix which would be easy to implement. (See also graeme’s post 4 on page 8 as to how the system could/should be checked by running results through algorithm again and again.)

Thus, you could feed the results for the past year in again and again, effectively simulating a century or so of the same results taking place on the same day of the year annually, look at what the ranking are at the end of it and, providing nothing unexpected happens, paste the simulated data for 2110 over the real data for 2010... I doubt it would take the person who wrote the program long to set up a test harness to feed the results repetitively into the ranking program or otherwise tweak the program to do this. It shouldn’t be too hard to write the resulting data over the current data either.

One option would be for the people working on the code now to be asked if they could try something like that while they are on the case? It won’t take them very long. If it goes horribly wrong and the data doesn’t converge it would be nice to know now. No amount of algorithm tweaking will fix this...

[graeme]

for the people working on the code now to be asked if

Or better still, just put the code and data in the public domain.

send me some recent examples of this effect?

If the algorithm has been properly rerun, recent would be no different to any other time.
Do you suspect it hasn't been done properly?

[awk]

But, as I'm sure I've said many times before, there's a temptation to put absolute faith in comparison of mins/km (adjusted or otherwise for height climb) when there's often little real justification for doing so!

Two courses at the same event could:

have different fractions of path run
have different ratios of runnable to slower terrain
have different ratios of optimal route to straight line route
have height climbs measured erroneously
have different ratios of controls to course length, hence different amounts of slower navigation in the circle
etc etc

I would argue that, rather than seeing where your mins/km would place you on a different course, it's more reliable to see where your rankings points would put you instead!

I ran 7.9k/160m/24c in 58.31 to score 1095 pts. At the same event, a clubmate ran 6.6k/125m/17c in 56.42 to score 1084 pts. All the indications I can see - comparison of common legs, comparison of similar legs, density of controls (I had greater), gradient (trivial, but if anything mine was very marginally greater), variations in terrain and technicality (virtually nil - but, if anything the shorter course was easier) etc etc, indicate that there is simply not sufficient in the possible reasons you give to explain why I am only 11pts in front of my clubmate (I was, admittedly, behind before the 'corrections' were put through!) having run the longer course in barely 2 mins slower than he ran the shorter, at something like 15% (one-sixth) faster speed.

Even if the distances etc aren't precisely accurate, a cursory look at the courses and at the times will tell you something is wrong with the rankings. It seems to me that rather truer is that There's a temptation to put absolute faith in comparison of ranking points when there's often little real justification for doing so!

[King Penguin]

re. Graeme's evidence that he scores less when running M21 than when running M45 :

I finally have a Black score, my previous Black runs all having been inquorate (see separate topic "are Black and national rankings incompatible ?").

My best 6 are all in the low 1200's.
My recollection is that the majority of my results in the last 9 months (Brown or Short Brown or M50 when entry is by age class) have given points in the high 1100's (would be nice to check by displaying all my scores but that facility has not yet been made available)
Last Sunday I ran 14.3 km Black, at 6.5 mins/k. I had a good run with no significant mistakes, just a few minor small wobbles (few seconds) close to the circle. I scored 1174 points.
The same mins/k would have given me about 1170 on Brown and about 1140 on Short Brown.

I faded slightly towards the end, probably losing between 2 and 3 minutes (the result of a stubbed toe and the night score I did on the Saturday evening). Had I been 3 minutes faster on Black I would have scored 1189.

This data samlpe is woefully inadequate to be statisticall significant, but if taken in isolation at this one event (Ashridge) it suggests the outcome was about right and within acceptable variations due to participant mix.

[DJM]

AlanB wrote:
Is it possible to state (briefly and for the non-statistician) how the algorithm has been changed, and what the object of the change was?

Does today's news article on the BOF website give sufficient info?

http://www.britishorienteering.org.uk/index.php?pg=home&action=news&id=MjEwNA==&rtn=aHR0cDovL3d3dy5icml0aXNob3JpZW50ZWVyaW5nLm9yZy51ay8=

[AlanB]

AlanB wrote:
Is it possible to state (briefly and for the non-statistician) how the algorithm has been changed, and what the object of the change was?

Does today's news article on the BOF website give sufficient info?

http://www.britishorienteering.org.uk/index.php?pg=home&action=news&id=MjEwNA==&rtn=aHR0cDovL3d3dy5icml0aXNob3JpZW50ZWVyaW5nLm9yZy51ay8=

Yes, thanks.

[Big Jon]

GIGO - garbage in garbage out.