ranking list

[SYO Member]

This is surely the solution to the problem, at least for non-Middle or Sprint Level 2s), of people who either deliberately (e.g coming back from injury) or inadvertantly run a course that gives them an unfairly high number of ranking points.

Noting that in the past for a Regional, a Short course for most age categories was always 2 courses below that of their Long, in the new world of colour courses for a Regional, don't give ranking points to anybody who runs more than 2 courses below their recommended Long.

Using M55 as an example. Blue is the nominated course for Long, Green is for Short. Don't be given Ranking points if you run any course less than Green.

There are one or two age categories that don't quite follow this rule (e.g. W20L/S, M75L/S) but these could be hardcoded into the algorithms.

[distracted]

I think some people need to give the system a bit of time to settle down before suggesting any major changes... it's only been going for just over a week!

[SYO Member]

You're right, it is early days, but my point is pretty fundamental.

I did ask the question about running up a few weeks ago and Greywolf replied that yes, it's OK to do so and still score points, but I'm surprised that significant running down could have slipped through the net, although I'm sure it's all been thoroughly tested.

But as you say, maybe as things pan out, it won't be a big issue.

[Scott]

Significant running down is going to remain a problem because of the way the algorithm trims "outliers" and ignores them in calculating ranking points. This is perfectly valid in principle, but the problem is that the algorithm regards an outlier as anyone taking over twice the winner's time. If someone is "running down", a large majority of the field can take over half the winner's time, and be discarded for the purposes of the ranking calculations. Greywolf explains it here, correcting my own slightly muddled explanation.

[graeme]

Significant running down is going to remain a problem because of the way the algorithm trims "outliers"

This problem, at least, is easily fixed by using an average rather than winner's time to set the cutoff.
Of course, it wasn't broken in the proposal which came from the event structure group.

Does anyone know how they started the lists? I know we discussed this at length already
viewtopic.php?f=1&t=8341&st=0&sk=t&sd=a&start=60 but I don't see it for the new system. If BOF botched the start up (e.g. giving S runners to many points), many of the other problems will go away in time.

[greywolf]

The seed data was based on seconds per corrected kilometre over a few major events, standardised to a mean of 1000 and SD of 200 - to calculate seed points for each individual.

This procedure has the potential for creating a systematic bias in that it regards an M45 running a 8km 200m climb course in 80 minutes as equivalent to an W60 running a 4km 100m climb course in 40 mins, i.e. it assumes that pace is independent of distance. (There may also be a systematic planning bias in that the courses for older competitors may be deliberately planned to be less physical - in the sense of avoiding rough and slow terrain - and therefore they are relatively quicker per km).

Note that this bias is in favour of shorter courses, and not “S” classes per se, so this advantages M55S over M55L, but M55L is advantaged over M21S etc.

It means that all those running shorter courses get a few (can’t quantify, sorry, but think it will be more like 10 than 100) extra points – the very high scores for course winners come more from the outlier issue.

However time should erode this effect if there is sufficient mixing of age classes in race results. Races with fewer courses (urban races, 7-course colour codeds etc) do this much better than BOC.

One alternative would be to introduce some sort of fudge factor by which double the distance = double the time + 10% or whatever – the effect would be to “stratify” (not sure is that’s the right word) the seed data more. So the overall range would be the same, but mean scores for individual age classes would be wider spread – so e.g. the mean seed score for W60 would be a bit lower – which means that the ranking points for W60s and all who run against them would be a bit lower.

Speculatively, it occurs to me that another alternative might have been to standardise the mean but not the SD of the seed data (i.e. that the issue arises from the underlying distribution)– but I haven’t seen the unstandardised data and my poor woodcutter’s brain can’t work out all the implications

[DJM]

Does anyone know how they started the lists?

Results data were taken from the following events in 2008: JK (all 3 individual days), BOC, FCC Final, Brown Clee National Event and November Classic. Using published values for course lengths and height climbs, mins/km speeds (corrected for height climb) for every competitor on M/W18+ courses at each event were then calculated for each day. Thus each of the seven events produced a list of several hundred names (nearly 2000 for JK2/3), each with a mins/km speed alongside.

These speeds were then standardised for each day so that the mean speed equated to 1000 points and the standard deviation to 200 points. Finally, an average for each competitor over the seven events formed the basis of the seed data used to start the lists. Some 2803 runners had points on this start-up list.

The rankings working party looked at many ways of producing seed data, some being mentioned in the thread Graeme alluded to, and came to the conclusion that this one had the fewest associated "gremlins"! We recognised that measurement of course length and climb is not an exact science and that some terrains make mins/km comparisons hard, but taking an average over seven events was designed to minimise any such discrepancies.

But we always recognised that the algorithm may need to be revised in the light of experience and have noted the issues raised by this nopesport thread. One of these relates to points gained from "S courses" and I have to confess myself to being embarrassed by the number I gained from "S" races last year ... I see my position dropping when we rectify this

[graeme]

Thanks Jon.

Like you say, its not a very good way to do it. It introduces exactly the biases people have noticed, but this should go away with time.
What I find disappointing is that in the proof-of-concept for the single list we* did it the right way (i.e. the exact science!), by running over historical data until the list became self-consistent, but for the actual implementation you* didn't.

* Pronoun alert...
"We" - not me, I just did the algorithm: Mike Cumpstey, David Rosen and Seamus Cunnane
inter alia did the hard work.
"You" - probably not you, Jon.

[EddieH]

Looking at the Edinburgh City results it demonstrates that it will take a long time (i.e. years) for the short course bias to be ironed out. Because of initial bias the short course points are clearly inflated over the long by quite some margin.

Jason scores 1209 points on the long and would have been well in excess of 1300 on the short had he been given a score for that.

He was haowever awarded 1274 on the yellow - I am not quite sure how he can score twice in the same event having apparently run all 3 courses

[EddieH]

Also at Edinburgh why do so many seniors not have a score by their name? - I thought perhaps they had not renewed their membership but I notice some cases where that is definitely not the case.

[Scott]

They will only score ranking point if the results file contains their BOF number. So some BOF members might not score points if they don't type their membership number into the online entries system, or if the organiser doesn't collect BOF numbers from EODs.

The organiser can always go through the results and fill in as many missing BOF numbers as they can identify in the database, but it is quite a bit of extra faff (took me about 20mins to do for the 2009 Oxford results).

[andy]

The organiser can always go through the results and fill in as many missing BOF numbers as they can identify in the database

excel vlookup (or similar) against the SI card list would certainly help considerably with this.

one thing the old system did (in its latter years) was attempt to match up people by means other than just BOFno. I'll certainly be making sure I am more on-the-ball with having BOFno fields populated in future

[Scott]

excel vlookup (or similar) against the SI card list would certainly help considerably with this.

I wish I had thought of that .

Another possibility might be to give people signed in to the BOF website a "claim this result" button, if the name and club for a run match those on their BOF membership card.

[madmike]

"claim this result" button, if the name and club for a run match those on their BOF membership card.

careful! don't forget we abolished second clubs on the membership card/database

[Spookster]

Another possibility might be to give people signed in to the BOF website a "claim this result" button, if the name and club for a run match those on their BOF membership card.

Can we have a "disown this result" button as well?