Can we determine a batsman's ability based on how he gets out?

I saw an interesting discussion online recently, suggesting that we could tell that some players had a better technique than others, based on how often they got out to different types of dismissals.

The theory was that players who get bowled or lbw have technical issues, while players who get out caught more often don't have those same issues.

This immediately stuck me as a multivariate statistics problem. Can we tell how good a batsman is based off the proportions of dismissals?

So I gathered together a sample of 160 players, all of whom had been dismissed in the 5 most common ways at least once each, and looked at what we could tell based off those players.

I grouped them based on batting average into 5 roughly equally sized groups. The groups were: Under 27.5, 27.5 to 37.5,37.5 to 43.5, 43.5 to 48.5 and over 48.5.

Once I filtered out players who hadn't played enough innings, I ended up with 29 from group 1, 30 from group 2, 28 from group 3, 36 from group 4 and  37 from group 5.

Their distributions were as follows:

There are some differences between the groups, but there seems to be more variation within the groups than between them.

I also looked at the raw numbers, without grouping, and adding in trend lines.

A pattern emerges - batsmen who have a very low average, and very high average tend to get bowled and run out more often than players who have an average between 20 and 50. Instead of getting out bowled, the players who have their average between 20 and 50 tend to get out caught more often.

This made me wonder if I could find some technique to group them effectively. I wasn't hopeful, because again the variation within the different groups seemed to be greater than the difference, other than from the very edge to the middle.

The methods that I chose to try were Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), Random Forest (categorical) and Random Forest (regression then rounding). I'm aware that most readers won't have studied multivariate statistics, so I'll briefly explain how these methods work. If you don't care how they work, click here to skip.

Linear and Quadratic Discrimination (LDA and QDA) can be imagined like plotting all the information on a giant multi-dimensional graph, then rotating the axes until the different groups are as separated as much as possible. The Linear version assumes that there are straight lines that separate the different groups, while the Quadratic version allows the groups to be separated by a curve. This is a more powerful technique, but it needs more data to be able to get an answer.

An example of how LDA works is in the two graphs below. There's a set of data that has two variables and is in two groups, and can't be easily distinguished by splitting any variable alone (displayed on the graph on the left). By making a decision based just on Variable 1, the best split gets 20 out of the 30 points classified correctly, and the best split on Variable 2 gets 22 out of the 30 points classified correctly. But if we put in a set of axes that are rotated, (the green lines) and redraw the graph (on the right) the two groups are able to be split quite well by being greater than or less than -0.45 on the new rotated x axis. By splitting on the rotated axis, 27 out of the 30 points are now classified correctly.

Random Forest is a technique that creates lots of decision trees, each based on small samples of the data and only some variables. Then it averages them out, giving the ones that performed best a higher weighting. This is a harder method to explain easily, but the process itself is actually reasonably simple. It's often remarkably effective for making predictions, but can be hard to explain how the final model actually works.

To test the four methods, I first of all tried doing leave-one-out cross validation to see how they performed. This is where I use all but one data point to build the model, then test that model on the remaining data point and see how well it was allocated - I do this for all 160 batsmen.

The groups are roughly evenly split, so randomly allocating the different batsmen to different groups saw about 20% get put in the right place. The different methods saw these results:

Method	Success
LDA	25.00%
QDA	28.13%
Random Forest (groups)	18.75%
Random Forest (average)	23.75%

I wondered if these were within the range of what I would expect from just randomly allocating batsmen to groups, so I randomly allocated the batsmen to groups 10000 times, and saw what the distribution of the number correct looked like. Below is a graph of that, with the 4 methods on it.

The method that worked the best here (Random Forest to predict averages) was still beaten by about 1% of the randomly allocated trials.

One of the issues with just looking at the proportion correct is that it doesn't tell us about how many are close to being right. I wondered how they would go if I plotted the actual group against the expected group, and found a measure of goodness of fit for it. The values for the goodness of fit here go from 1 down, where 1 is a perfect fit, 0 is the result of just giving every point the average, and negative values are even worse than that.

Here's how the 4 methods stacked up:

Method	Goodness-of-fit
LDA	-0.710
QDA	-0.505
Random Forest (groups)	-0.624
Random Forest (average)	-0.064

The Random Forest (average) method was the best, but was still not as good a fit as just allocating every batsman to group 3. (That had a goodness of fit value of -0.009). 

I put these methods against a random allocation, to see how well they actually did:

Even though the Random Forest average method was better than random arrangement, it was not as good as just classifying everybody as group 3. 

What this tells us is that this data really is not very useful at all for making a classification. We know that the methods would all have done a reasonable job of distinguishing between the different groups if there was actually a difference between them that could be found. However, there is not any real way to distinguish how good a batsman is by the way that they got out.

I also tried some other methods, that theoretically shouldn't have been as good, just to see how well they went. I managed to get a couple that had a goodness-of-fit as high as -0.235 with 26% correct classification. These seem to be reasonable, but they were still not as good as just classifying every batsman as group 3. Any method that isn't as good as that, is really worthless in making a classification.

In conclusion, looking at the proportion of ways that a batsman has been dismissed is not particularly helpful in deciding how good they are. The differences within groups are much larger than the differences between groups. What that means is that if you're having an argument with someone on the internet, and they say that a player should be dropped because they get out LBW so often that it shows that they have a bad technique, you can smile smugly to yourself, knowing that they are speaking nonsense. You could even send them a link to this article if you want.

A quick look at the DRS rule with hawkeye and lbw

There is a significant issue with the way that hawkeye is used for DRS.

There is some doubt as to the exact position of the ball when captured on camera. It's only accurate to the nearest 2mm or so. While that's very accurate, once it's used to create a model, it can be dangerous. As a result there is a margin for error. Then there can be difficulty determining exactly where the ball hits the pad, especially where it brushes the front pad on the way to the second. This means that there is some doubt as to what the actual position of the ball is.

To overcome this, the ICC have ruled that more than half of the ball needs to hit the centre of the wicket. This is a user friendly option at first glance. The boundary is really clear, and the batsman needs to be clearly out in order to be given out. But near the boundaries there are occasionally situations where the ball is clearly going to hit, but instead the hawkeye system calls the ball "umpires call."

This is particularly ridiculous when the ball has hit the batsman on the back foot. In a situation where the ball has only an extra 40cm to travel, if the middle of the ball is just outside the middle of the stump then for the ball to miss the stumps, then the model would have to be out by 5.5 cm. On a distance of travel of 40cm that's allowing way too much margin for error (realistically there would be a significantly less than 1% chance of the ball missing the stumps).

A solution would be to look at a cone that was using a realistic model for the uncertainty. That would be more sensible for the commentators, fans and players to understand, and would actually provide a more sensible answer to the question "would the ball hit the stumps?"

I've put together a short video to demonstrate what I mean as well.

Shane Watson and The Form Myth

Shane Watson. Out lbw. Again.

I read somewhere recently that Shane Watson is in bad form.

They implied that he needed to go away and "discover some form." The more I am involved with cricket, the less I believe in form.

Shane Watson has a technical deficiency. He moves his front foot too far across when he is playing in-swing. He also clears his front leg against left arm spin, and plays across the ball. These technical problems aren't really an issue if he's playing against lower quality bowlers, because he can hit the ball, and so he doesn't get out lbw. Once he faces bowlers who can bowl the ball in the places where he finds it difficult to play, he gets out. The champion of cricket on YouTube, Robelinda2 has put up a compilation of some of his lbw dismissals here.

It seems that it often happens once he starts to attack the bowling. It means that his larger innings tend to get cut short even once he's underway. That's why he was labelled by one commentator as Wall Street (great in the twenties, crashes in the 30's and 40's).

I compared Watson with all top 7 batsmen in the last 3 years on their likelihood to get out lbw. Most of us have heard that Watson gets out lbw more than anyone in history except Junior Murray (min 30 matches), but when he gets out is particularly interesting.

I looked at the relative frequency of dismissal by lbw in innings under 30 and in innings 30+. I compared it to other batsmen in the last 3 years. (I picked the last 3 years to include the DRS factor) Here are the numbers:

Score	Watson	Other Batsmen
0-29	33.33%	20.48%
30 +	28.95%	13.78%

Watson is 63% more likely to get out lbw at the start of his innings than other batsmen, but he is 110% more likely than them to get out later on. It is a technical issue, not a form issue. Going away and playing some lower quality bowlers will not fix it. He needs to work on where to put his feet and how to play the different deliveries that are getting him out.

If he does fix that technical issue, the commentators will all say that he's in great form.  But perhaps it will just be that the bowlers are trying to bowl the ball to him in a way that he is now equipped to deal with.