Over the past few years, the application of vision and advanced imaging technology in sports has grown with the rapid developments of vision technology.
Although the best-known applications tend to be those highlighted by television, such as ‘Hawk-eye’ and slow-motion replays, vision has a host of other applications, including: the development of sports-related equipment, such as sports shoes and Formula 1 racing car tyres; and the analysis, for training purposes, of sports stars (not necessarily human!) in action. The use of slow-motion replays has already been adopted by American football, rugby union, and international cricket to help with refereeing and umpiring decisions, and there is a strong voice for the use of video technology in Association Football to determine whether the ball has crossed the goal-line.
Clearly, slow motion replays and multiple views from different angles allow incidents to be examined in greater depth than would be possible by a single referee or umpire, but purists argue that taking the ‘human element’ out of the decision-making process contravenes the ‘spirit of the game’ and that frequent stoppages for video referral will interrupt the flow of the game, extend the time elapsed for a match, and irritate paying supporters. However, whatever one’s particular viewpoint on these issues, there is no denying the vast capabilities that vision technology brings to sport. Broadly speaking, the applications fall into three main categories: the use of vision systems and image processing to calculate or track the trajectories of moving items, such as balls and people: high-speed imaging to allow frame by frame motion analysis; and the use of infra-red imaging to reveal information that could not be obtained from conventional cameras.
Geometrical systems
One of the best-known systems for helping with marginal decisions in sport is Hawk-Eye, from Hawk-Eye Innovations Ltd. The Hawk-Eye officiating system for tennis made its grass-court debut at the Queen's Club, UK, in June 2007 at the re-named ‘Artois’ tournament, following 18 months of trials. Also used at Wimbledon this year for the first time, the Hawk-Eye system is being officially used to allow players to challenge umpiring decisions. The Hawk-Eye officiating system has been developed in close co-operation with a member of UKIVA, Firstsight Vision, for the supply and integration of vision technology. The system uses 10 cameras to track the movement of the ball (Figure 1) and players, then a sophisticated image processing system calculates the trajectory of the ball to provide a definitive decision on whether the ball is in or out. Vision processing is used to identify the centre of the ball within each frame of each camera. Camera movement is compensated for by also tracking the lines of the court. The system triangulates the information from each calibrated camera to provide a 3D position of the ball. Time is taken into consideration by repeating this process for each frame, so that the 3D positions of ball can be combined to produce a single trajectory for the flight of the ball. The trajectory is then used to calculate the exact bounce contact area of the ball with the court. The system takes into account the skid and compression of the ball and achieves accuracies of 2-3mm. Hawk-Eye has also been a popular feature on TV cricket coverage since 2001, where its ability to track and project the trajectory of the ball has been used to analyse unofficially umpiring decisions for ‘leg before wicket’, as well as to showcase other features of the game, such as accurately comparing bowlers’ speed, swing, spin, line, and length.
Figure 1: Impact of a tennis ball on the court line. Courttesy Hawk-Eye Innovations.
The Department of Computer Vision at Leeds University has also been involved in tracking applications in sport. Research was originally carried out to track the movement of vehicles on the road. However, by teaming up with Sports performance analysts ProZone Ltd, the system has been adapted to track professional footballers and has been used by a number of Premiership clubs including Arsenal and Chelsea to help plan tactics and develop their training to suit players’ strengths and weaknesses. The software allows the positions of the footballers to be tracked from a number of views through video sequences, by modelling the ‘background’ and picking out areas where the colours change rapidly – the players – as ‘foreground’. This system is much faster than existing tracking techniques because it requires minimal manual intervention from ProZone. Managers are given statistics – including how fast a player moves, who passes to whom, and where each player is on the pitch at any point in the match.
High Speed Analysis
High-speed cameras operating at up to 1000fps and with high spatial resolution (1280 x 1024 pixels) (or at higher frame rates and reduced spatial resolution) allow complex movements to be filmed and slowed down for analysis. In one such application, Lambda Photometrics supplies high-speed lenses and cameras to ONTRACK High Speed Video Consultancy for the analysis of racehorses in motion, to make it easier to assess their dynamic movement visually. It is a far cry from the work of Edward Muybridge, who, in 1878, successfully photographed a horse in motion, showing that all of a horse’s hooves are in the air at the same time during a trot. Muybridge used a series of twenty-four cameras arranged along a track parallel to the horse's motion, with each camera-shutter controlled by a trip-wire triggered by the horse’s hooves. Modern high-speed imaging has far-reaching applications for training, treatment, and farriery. The legs of a racehorse are naturally moving very quickly during a full-speed gallop. The stance phase (time spent in contact with the ground) of a forelimb of a galloping racehorse is about a tenth of a second (0.1s). A high-speed camera operating at 500 fps captures about 50 frames during this time, compared to about 5 frames from a standard camera, thus providing more information on the movement. Imaging in this way can reveal a great deal regarding the loading of the joints in horses. One recent investigation showed that pastern joint displacement occurred in the hind limb of a horse when landing from a jump. This happened in 0.004s and was seen only because of the high frame rates. The high spatial resolution also allows observation of muscles contracting and extending, and the ligament and tendons being loaded. The technique can also be applied to human sports activities, including analysis of athletes’ techniques and golf swings. Vicon, part of OMG plc, offers high-resolution, high frame-rate camera systems that record the position of retro-reflective markers attached to the sportsman. By extracting 3D information, using advanced computer vision technology, specific biomechanical modelling such as actual joint angles and forces can be displayed in real time as the athlete is moving. These real-time calculations can be overlaid with historical data for comparisons. With real-time feedback, the athlete can adjust his or her movement patterns to immediately see the effect of the change and thus improve performance.
Infrared imaging
Infrared imaging has a number of applications in the sporting arena. Returning to cricket, Cedip Infrared Systems, working with BBG Sports (Melbourne, Australia) has developed a system called ‘Hot Spot’, aimed at eliminating speculation surrounding a cricket dismissal. Trialled at the first Ashes cricket test match between Australia and England in Brisbane in 2006, by broadcasters the Nine Network and more recently by Sky Sports during the Test series between England and the West Indies, the ‘Hot Spot’ uses two IR cameras positioned above the field of play at opposite ends of the ground. The impact of the ball against any part of the batsman (bat, pad, leg etc) results in the generation of a localised heat spot, which can be seen as a bright spot on the IR image (Figure 2). This shows whether the ball has hit the bat or not and could be used in adjudicating disputed catches or leg before wicket appeals, if it is suspected that the ball has hit the bat before the pad.
Figure 2: Infrared image of cricket ball hitting edge of bat and pad. Courtesy Cedip Infrared Systems
Flir Systems has also been involved in sports applications, with the latest running shoes for athletes and Formula 1 racing car tyres. Its infrared cameras have been used by Adidas in the development of the ClimaCool2 ventilation and cooling system in athletics’ footwear. When athletes run, their feet sweat, which causes discomfort and can prompt the development of blisters. The camera’s prime function was to see which part of the foot got hottest whilst running, replacing measurements at several pre-determined spots on the shoes using temperature sensors. Blur-free imaging and the real-time measurement of small temperatures changes (0.02¢ªC) when the athlete is running at full speed showed how the shoe was reacting to the temperature differences in the foot over a period of time (Figure 3). 
Figure 3: Thermal image of heat distribution in running shoe. Courtesy of Flir Systems.
The need to predict the thermal performance of tyres is of vital importance in the Formula 1 racing industry. During a Grand Prix race, a tyre rotates about 150,000 times with a vertical load in the region of 500 kilos. Infrared technology is an ideal method for showing the thermal performance of tyres under race conditions and in different weathers. Infrared cameras can also be used to provide a detailed understanding of how the complex materials used in disc brakes react under load, and how heat is dissipated in exhaust systems.
The money factor
A phenomenal amount of money is invested in professional sport and anything that gives a player, club, or sports presentation organisation a competitive edge will be highly sought after. There are a several systems under development involving vision technology aimed at offering just these sorts of benefits. The promise of healthy profits is a powerful incentive for the vision industry.
Don Braggins is director of the UK Industrial Vision Association (UKIVA)
Acknowledgements
The author would like to thank UKIVA members Cedip Infrared Systems, Firstsight Vision, Flir Systems, Lambda Photometrics, OMG, and the Department of Computer Vision, Leeds University for information used in this article.
