Back in the 1980′s Jan Carlzon was trying to breathe new life into an ailing Scandinavian Air Services. He was famous for saying “You cannot improve one thing by 1000% but you can improve 1000 little things by 1%”. Wind the clock forward 20 years and “Team GB” scooped a helmet-ful of gold medals in Beijing by following a similar principle. According to David Brailsford, the British Cycling Performance Director, their success has come by way of the “aggregation of marginal gains”.

A while ago I utilized the principle to determine that I needed to create my own ratings and improve them over time.

Empirical analysis over the last 6 months suggests that the ratings are profitable however trying to squeeze an additional 1% of improvement out of the ratings was becoming detrimental to their current performance.

I am considering altering the ratings based on some interesting methods outlined in the book “Who’s #1 – The Science of Rating and Ranking” and some ideas I have kicking around in my head to use Monte Carlo simulation to aid my statistical models in handling chaos (the one factor that’s hard to numerate in horse racing) but frankly, there is only so much tinkering you can get away with before the process can be consider to be “good enough”!

So it’s onto the next marginal gain, full automation of my betting process.

Currently, quite a bit of the ratings process is computer driven under Excel, however there are some aspects of the process that require human intervention and would be beneficial if automated. For example, absorbing the live Betfair market into the ratings real time.

My first automated software called “Bender” (after the robot character from the cartoon series Futurama) was implemented in 2003 and its function was to calculate the speed ratings based on the days racing.

The “Bot” would on a daily basis scalp the data horse racing results section of the Teletext web site and create a database of individual horse speed ratings that could be read by another piece of software that would allow me to slice and dice the data so I could answer the question “who was the fastest horse given the prevailing conditions of the race?”

Unfortunately due to budget cuts within Channel 4, the horse racing section of the Teletext web site was axed in 2005 and the data source for Bender was no more.

Today’s Bender (2.0) has a different function and has two modes of operation

click on image to expand

Without Proform Professional the act of generating the ratings would be difficult as the software provides the source data for my statistical analysis model and presents the data in a manner that’s easy for other applications to extract what’s needed.

Currently the data required is cut and pasted into an Excel spreadsheet by mouse clicks so an obvious improvement would be to extract the data automatically via SQL queries.

In standalone mode and using Excel Visual Basic for Applications (VBA) to take care of the number crunching (and allowing for any minor tweaks to the model via ?), the ratings and decision matrix (read betting forecast and sharp ratio calculations) are created in preparation for reformatting prior to automatically posting the outcomes on slipperytoad.co.uk. This will save me a lot of time.

In automated mode, the decision matrix can form the basis of automated bets into the Betfair market. Today this is a manual process however, I would like to automate this process and let Bender “bet the numbers” based on perceived value in the market as identified by the statistical model minutes before the start of the race.

I would also like to expand my betting arsenal into automated Computer Straight Forecast / Exacta’s which currently are manual bets and labour intensive and therefore limited to on course visits only.

The Wager Engine can be commanded as needed with appropriate stake

After much research and given the wealth of examples on the Betfair Developers Network, I have decided to code the application under VBA using Microsoft’s Visual Studio Express 2010.

My plan is to use the lull in the All Weather season during the summer to architect the software in time for live firing in November.

Any pointers or words of advice welcome.

Result

A “hd” between top rated glory and defeat!

Result

The show piece event of the All Weather season, the running of the Winter Derby has produced a familiar pattern over the years. I see minimal reason to alter my thinking.

Since taking the decision to focus on the All Weather Racing, I’ve kept a keen eye on events in Dubai and more importantly the form of runners when returning to the UK.

Let’s just say that Emerald Wilderness, Lyssio and Sooraah did not make the shortlist although the latter is respected.

The fact that Suits Me is a non runner tells you everything you need to know about the draw bias in this race. Pre withdrawal, as the only front runner, there was more chance of me being an underware model that him getting the lead from stall 14.

Unfortunately, the problem for us punters now is that the pace dynamics are thrown wide open with no identifiable pace in the race ….. Messy (and I’m not talking about the number 10 who plays for Barcelona) and suggests that come 15:30, we could be all scratching our heads at the outcome.

Cai Shen and Premio Loco have difficult but not impossible tasks from their stall positions.

So calibrating the ratings with the computer that sits between my ears and the draw bias suggests a big run from the progressive Junoob as the form of Riggings indicates that even with a predicted slow pace, his stamina is questionable.

Of the rest drawn low, the Fahey trained Myplacelater also makes the shortlist however the record of northern raids in this race is poor.

“Here be dragons” is a phrase used to denote dangerous or unexplored territories, in imitation of the medieval practice of putting sea serpents and other mythological creatures in uncharted areas of maps

I’m sure there is a punter out there that would disagree, but “Leafy Lingfield” is one of those courses (like Bath) that bewilder most punters making it to coin a phrase “bookie friendly”.

I am not one to believe in the fickle hand of fate however, like the ancient cartographers who annotated unknown areas of their maps with warnings, once you enter into boundaries of Lingfield, the gambling gods or some strange beast devour the bets of punters.

Why? Let’s start with the course.

Unique in its layout Lingfield is not flat like the other All Weather venues but undulating.

The best way to describe its configuration is to imagine a snooker ball triangle tilted on one side toward you and the left hand corner of the triangle tilted slightly upwards.

As you can see from the diagram above the bends are tight and from the highest point (near the 5f start) it’s downhill toward the finish.

Because the track is not flat, horses can conserve energy on the downward legs of the course and therefore a horse’s stamina is not questioned at Lingfield. This requires the punter to alter his/her thinking when assessing the merits of a horse at certain distances. Another X factor in the puzzle that is Lingfield.

Most All Weather races up to 8 furlongs are ran at a fair clip with horses that like to prominently winning the majority of races.

Based on the results categorised by pace preference (sourced from the excellent Proform Professional Database), the Impact Statistics at Lingfield suggest this is the case.

However, a whole string of Barry Dennis losing betting slips and my eyes tell me that typically the horses that hit the front seem to “set it up” for something running off the pace suggesting that if you are a horse that tries to do it from the front you might as well paint a roundel on your behind as demonstrated by the following race ran over 6f.

Ah, but can we not just apply a bit of pace handicapping to solve the conundrum which is Lingfield? Not so.

I have witnessed pre-race hold up runners all of a sudden decide to win from the front, I have seen horses identified at hold up horses win from the back, I have won backing front runners who make all over the non sprint distance of 1m2f (thank you Sri Diamond, Winter Derby 2006) which suggests that it is not easy to get it right.

“What gets us into trouble is not what we don’t know. It’s what we know for sure that just ain’t so.” [Mark Twain]

Based on a Twitter debate a while ago and again using data from Proform Professional from 1997 at Lingfield, I analysed the correlation between the going allowance and the total winning distance for the meeting. The objective was to see if factoring the pre-race weather conditions i.e. cold snap or wet spell gave me an edge in predicting the likely winning distance makeup when spread betting. 

The statistics didn’t yield much however given the clustering the results indicated that the clerk of the course and his track team keep a tight rein on how the track rides at Lingfield. The data also suggested that the winning distances are clustered around an average of approx 9 lengths, not much for your typical card of 6-8 races.

After analyzing the data, I now know why they call the furlong off the final bend at Lingfield “the longest in racing” with the lead changing more times than the finish in a virtual cartoon horse race shown in your local bookmakers.

As they run down the hill, could the reason be that horses are switching their lead leg off the turn? Could the reason be that horses have used up too much energy maintaining their position at the [Front|Middle|Rear] of the pack? Could the reason be that the jockeys find it difficult to judge the pace in the race? Could the reason be that a lot of poor races are carded at Lingfield adding yet more chaos to the problem space? Could the reason be that horses on the inside rail get cramped for room and therefore lose momentum?

Whatever the case, a punter be it fixed odds, spread or in-running can get lulled into thinking they’ve won bundles and start counting the moola only to have their hopes dashed and lose it all in microseconds due to those thrilling Lingfield close finishes.

So with the draw statistics indicating “fair” with the exception of a few wide drawn berths over certain distances, in my humble opinion (and I’m open to different perspectives) given its layout, Lingfield’s pace dynamics and the pace within the race are a major factor in deciding the outcome.

This presents a difficult challenge for even the most experienced punters so the recommendation is to tread carefully when betting at Lingfield.

Those of you who have been following my ratings have noted a steady improvement in their accuracy, although I am still not satisfied.

For a while now I have been trying to get my head around William Benter’s seminal paper “Computer Based Horse Race Handicapping and Wagering Systems” (Efficiency of Racetrack Betting Markets) where he outlines a practical and successful computerised horse race handicapping and wagering system he developed. I’m not a mathematician or statistician so I needed to develop a basic understanding of these skills to develop my own version of his model; hence another reason for not blogging recently.

One major area of discontent with my model was the accuracy of the final rating and how it translated to winning probabilities. This is function of the limited factors I was using in my model and as such I had many failed attempts at creating a semi accurate betting forecast which also factored in the favourite longshot bias. The majority of computer generated tissues do not take account of this bias and in my mind fundamentally flawed.

In addition, the other area of contention was that even with most sophisticated model on the planet, I could not hope to include all handicapping factors into the model that out performed the crowd’s powers of prediction thus

The chart is old however you can create a modern day version if you take horse racing results over a significant period and create a scatter plot of observed proportion of races won by horses verses the probability implied by their betting odds. You will see that the “fit” is remarkable; an R squared of around 0.98. What does this chart mean to non stats junkies?

The Public Is Very Good At What It Does
Not only can the public tell the difference between two low odds horses say a horse that should be even money (1/1) versus a horse that should be say 2/1 we might be able to do that at a decent level, but it can tell the difference between high odds horses say a horse that should be 20-1 versus a horse that should be 25/1.

Now how do you do that? They are indistinguishable in the form yet the public can separate them in the long run. The horse that is 20-1 will win more than the horse that is 25-1.

If you are not convinced by the predictive powers of the crowd take a moment to watch this video.

Benter recognised the public’s implied probability estimates generally correspond well with the actual frequencies of winning. He realised that what was needed in his model was a way to combine the judgements of two experts, (i.e. the fundamental model and the public).

After thinking and modelling different approaches to absorb pre- race market opinion into my own model, I have decided to implement one from today and try it out over an expended period of time using real races. I have the old model to revert to if everything goes Pete Tong, although my gut feel tells me that I must embark on a modification of the current model to increase its accuracy.

Southwell is the black sheep of all the All Weather venues in that it has one major difference, the fibresand surface

As fibresand is a mixture of sand and wispy fibres it is sometimes referred to as “spube”, this being derived from the “s” of sand and the resemblance of a kind of human hair.

The racing is also of significance. The now unique fibresand racing surface is quite deep and so makes the track a good stamina test. Ability to stay the distance and fitness are therefore big advantages at Southwell..

In respect to punting pointers, I had already outlined a few in a previous blog post based on a course visit a while ago. Using the power of Proform Professional I’ll include additional context to this blog entry.

Let’s begin with the most prevalent punting factor at Southwell; pace. When pundits hark on about the fact that Southwell is a pace oriented course, here’s why:

As inferred from the stats, up to 8 furlongs horses that like to run prominently (“L”) have a distinct advantage over their “hold up” counter parts. Once you are out in front at Southwell, the “deep” fibresand surface does not allow you to easily peg any advantage gained from being fast out of the traps. To replicate the effect, next time you are on a beach, offer to run against an opponent of lesser ability over 100 meters, but give them a 5 meter head start.

From a punting perspective a multitude of betting options are on offer using this one factor alone; for example laying the favourite based on its running preference.

In respect to draw stats, like most oval course configurations the closer you are to the inside, trigonometry suggest the less distance that has to be run and therefore the greater advantage a horse has if it’s close to the rail. However watch out for stall 1.

Although counter intuitive to my above statement, there’s something in the manner how the rail rides and at certain times of the year rendering stall 1 a “coffin” box.

There are a number of reasons why this is the case which I won’t regurgitate. Hopefully in the next few months I’ll get the definitive reason from the friendly bunch that is the Southwell regulars at my next course visit.

Simply, the inner rail (and outer over 5 furlongs) rides deeper than other parts of the track which saps the stamina of horses who due to their berth at the off, try to hold their inside position prior to the first turn.

So, rock … meet hard place because if you are in stall 1 in one turn races at Southwell and run prominently you are done for, miss the break and you are done for plus sucking in kickback off the hoofs of the horses in front of you for added measure.

5 furlongs sprints at Southwell also presents a different puzzle for punters. Firstly it’s unique in that the race is on a straight track where at Lingfield, Kempton, Wolverhampton and Dundalk have at least one turn over 5 furlong sprints. Secondly dependant on the number of runners the rails are positioned either “inner” or “outer” which renders all form book 5f Southwell draw stats meaningless.

The trick for 5 furlong races is to first determine stall position and note the pace horses whose stall position is aligned toward the middle of the track and dependant on number of runners in the race be cautious of horses drawn toward the rail either side of the track.

Another tip I picked up from reading Nick Mordins book (Betting For a Living) is to side with horses that have a “straight course” form pattern in their past performances.

Finally horses that favour fibresand are a subset within a subset that perform well on All Weather surfaces, sires to consider at Southwell are:

In summary Southwell uniqueness suggests that discerning punters who wish to dabble in All Weather races should go prospecting at Southwell. There’s gold in them thar fibresand.

Mr X, an old work colleague and I are attending the final meeting of 2011 at Bath. Obviously bets a plenty as we all like a punt

During the preparation of my battle plan, noticed some interesting insights from the blokes at Sporting Index.

STOP ON WINNER: SELL 39 – 42 BUY
Suggests the first 3 favourites are vulnerable but the jolly in the 4th looks nailed on. A possible angle might be to BUY then look to bail out when the spread is adjusted in the 15:05

Settled @ 20

DISTANCES (F): SELL 11.5 – 12.5 BUY
My calculations suggest much tighter margins between first and second over the 8 races today. SELL

Settled @ 10

DANE O NEILL(7) SELL 31 – 34 BUY
SILVESTRE DE SOUSA (6) SELL 31 – 34 BUY
With 25 points for a win and 10 points for 2nd suggests that the spread Tefal heads don’t expect the mounts ridden by these jocks to do well. I have factored this into my decision making process.

Dane O Neill (7) Index Settled @ 45
Silvestre De Sousa (6) Index Settled @ 50

Ratings

13:30

1st Carinya (IRE) 4/1jf

14:00

1st El Dececy (USA) 9/2fav

15:05

1st Hidden Valley 8/1

16:10

1st Blazing Field 8/1

16:40

1st Tegan 4/1

17:15

For those who are betting in this race we salute you!

1st Seeking Magic 11/4fav