How’s your football team doing? I can tell you, if you’re a Bayern Munich fan you’ve got to be pretty pleased with their performance, with 15 wins, one loss and three draws in the Bundesliga this season. Not only that though – I can tell you that, in their most recent 2-0 win over Stuttgart at the weekend, the team won 57.7 per cent of tackles, made 461 complete passes (to only 165 for Stuttgart) and 765 touches (Stuttgart made 431), had 64 per cent possession and made 14 attempts on goal to Stuttgart’s five. It’s not that Stuttgart weren’t trying, either; they were no slouches covering 114.9km as a team over the 90 minutes compared to Bayern’s 112.4km, making 570 intensive runs (Bayern: 493), 399 fast runs (Bayern: 319), and 171 sprints (Bayern: 174). I can even break those stats down further by player and tell you that Stuttgart’s William Kvist ran the furthest during the match, covering 12.06km.
These match statistics come courtesy of German company Impire and its Vis Track device, a vision system installed at football stadiums of teams playing in the Bundesliga and 2nd Bundesliga, Germany’s first and second divisions. Vis Track uses two cameras to track each player over the course of a match in real time, as well as tracking the ball. Analytical information such as the tactics of each team and the distance run and turns of speed of individual players during the game can all be generated from the data, all of which is distributed via Impire to the various newspaper, TV and media channels reporting on the games. The teams themselves will also study the tracking data, for analysis of their own players’ performances as well as for scouting purposes.
Vis Track sends video streams of the match to four PCs: one records the game for playback later; there are two manned stations whereby the live feeds from each of the cameras is monitored during the game; and a fourth generating a stitched image, covering the entire pitch.
Stemmer Imaging provided Impire with components and expertise when building its system. Martin Kersting, technical director at Stemmer Imaging, explains the necessity of having a human sat controlling the system throughout the game: ‘In a situation where two players are very close together, the algorithm might fail to track each individual or confuse which player is which. There’s always an operator sitting at the system to ensure each player is tracked accurately.’
The complete view of the pitch stitched together from the two aligned cameras produces an image that’s full HD in height and twice the width of full HD. From this virtual panoramic view broadcasters can frame HD video, wherever the action is taking place on the field, across the two feeds. ‘You really have the feeling of sitting in a chair which is moving across the field,’ says Kersting.
The cameras are permanently installed at each stadium and are a feature not only in Germany but also in Spain, as well as at different clubs around the world, for training purposes.
Impire also generates what’s called a ‘heat map’, which is the amount of time individual players spend in a given section of the field. ‘Suddenly you can see that one team attacks a lot down the right wing, for instance, so it adds a lot of analytical data to team tactics,’ says Kersting. There are also services to send this information directly to team coaches, if a team owns the system, so they can get immediate feedback about their performance and their opponent’s performance.
The system makes heavy use of GPU and multicore computers; Stemmer Imaging provided GPU image processing software and expertise. The server is recording raw data and the whole system can be changed into an offline system for video playback after the match.
Using a multiple number of PCs means the image data has to be distributed in a multicast mode. Kersting explains the importance of synchronising the feeds: ‘You need to ensure the system captures two images from the cameras at exactly the same time. Otherwise all the position and speed information would be useless.’
Germany’s Bundesliga makes use of Impire’s Vis Track system to track football players and generate stats on the games. Credit: Impire and Stemmer Imaging
The system uses Stemmer Imaging’s GEVision Server (a GigE Vision camera simulator) to play back the image data of a previously recorded game to the multiple PC’s synchronously. GEVision Server turns the PC playing back the footage into effectually a GigE camera and the server receiving this data will not recognise any difference between an actual camera and the camera simulator. The timing must be synchronised so that recording and playback speeds are the same right down to the individual frame. The broadcaster might want to jump to a scene from 69 minutes 12 seconds, for example, and the timing between the recording and playback has to be the same to do this.
‘In 2011, when the systems were first introduced, there was a huge discussion in the German press about whether to publish this information,’ recalls Kersting. He says that once the stats were available though it suddenly became very clear which players cover most ground and which are faster than others. All this information is available to fans and supporters.
Sporting prowess
Video analytics is becoming more widely used in sport as a tool to improve a team or athlete’s performance. Belgian company Camargus International has developed a video solution for sports analysis. Like the Vis Track system, it can produce a virtual, stitched image, this time from multiple cameras, from four to 40 depending on the scope of the project, says Sacha Vekeman, VP at Camargus. The user can then zoom into and out of the virtual image. The system makes use of Basler Aviator cameras and combines the camera technology, storage hardware and software for an end-to-end sports analytics video solution.
Vekeman gives an idea of the capabilities of the system: ‘We can capture a football match, the full 90 minutes, and not only from one camera viewpoint, but the entire field; all the movement, all the players, all the audience is recorded, and this is available through virtual camera control in real time or via a replay function.’
The system is suitable for live broadcasting, but the other growing market, according to Vekeman, is analysing sporting performance. ‘Each nation wants better athletes,’ he states. ‘They need video data on training and matches, all of which they want to use for analytical interpretation.’
Camargus’ system has been used by the Dutch national swimming federation to capture images of training and to track arm and other movements and then feed statistics from the data back to the trainer. ‘Previously, the trainers had to use handheld cameras to film the swimmer and it took two days to write a summary report about the athlete’s training. Today, with this system, analytical data can be generated in almost real time,’ Vekeman says.
A stitched panoramic view of an American football match, generated with a video analytics system from Camargus. Credit: Camargus International
The Basler Aviator cameras used in the system provide 2 megapixel resolution capturing images at 60fps. The server can process and store terabytes of data and also has the necessary computing power to stitch to broadcast quality real-time video. ‘These systems are permanent installations and the imaging hardware needed to be reliable to minimise maintenance costs. This is why we chose industrial imaging components and the Basler cameras,’ Vekeman says. ‘The system is all built around simplicity of use, limiting the number of options a user is faced with in order to set up and run it,’ he continues. ‘Having said that, where we do open up the system is in terms of the API. Using this, software engineers can develop specific software for track-and-trace and sports performance and analytics.’
While professional sports teams and clubs might be able to invest thousands of Euros on a camera system, the average individual doesn’t have that kind of capital. There are, however, systems out there based on imaging designed with the public in mind. Golf swing analysis equipment is a good example; gait analysis for runners is another, even to the point of getting a free video analysis at sports shops when buying a pair of trainers.
Running man
The footage recorded by a sports shop, however, is very different to the technology employed to undertake a full scientific analysis of a runner’s gait. Run3D, a spin-out from the University of Oxford, aims to bridge this gap between scientific analysis of a runner and simple video footage.
Run3D provides a computerised 3D assessment of a runner in an attempt to identify any abnormal patterns in their gait and, through correcting their running style, prevent injuries. A full analysis with Run3D costs £295, a 10th of a scientific analysis from the Nuffield Orthopaedic Centre, points out Dr Jessica Leitch, CEO of Run3D, (she quotes an assessment from the Nuffield Orthopaedic Centre as costing around £3,000).
‘Gait analysis performed in this way has been around for 10 to 15 years,’ she says. ‘What’s different is that this system is, firstly specific to running, and secondly it’s accessible to a much broader range of people.’
Run3D has three infrared cameras viewing the back of the runner, with markers positioned on the back of the legs. ‘We use the same technology as the computer animation industry – infrared cameras imaging markers positioned on the runner,’ explains Leitch. The markers are placed on known anatomical landmarks and used to define a 3D coordinate system in each section of the body. The person is modelled as having a pelvis, two thighs, two shanks and two feet. Each segment is treated as a rigid body. The user defines a 3D coordinate system for each segment using the positioning markers. The relative movement between two segments about a joint will tell the runner what’s happening at that joint.
The system was initially opened at the Nuffield Orthopaedic Centre in December 2011. Through Isis Innovation, the technology transfer office of the University of Oxford, Oxford Technology Management invested in the system and it’s now located at the Oxford University sports centre, although the system is portable.
‘We’re the only facility in Europe able to offer this level of service,’ states Leitch. ‘Our gold assessment is a two hour assessment plus a 30-minute follow-up, and involves not just measuring your biomechanics, but measuring all aspects of your muscular strength, anatomical alignment, and flexibility. You can combine all that information and determine if the runner is twisting their hip too much and their knee muscle is weak, for instance, and we know that one controls the other so if we give exercises to strengthen those muscles then this will improve the biomechanics and stop you getting knee pain. It’s probably the most scientific approach to running injuries that is currently available.’
Strength and flexibility are measured quantitatively and the results are compared to a database of values, as are the biomechanical data, in order to get an objective reading. ‘It’s a matter of putting values to a runner’s movements and analysing their gait scientifically,’ Leitch adds. ‘We can say this muscle is weaker than average from our database of uninjured runners, so we need to strengthen that muscle.’
It’s incredibly difficult to pick up the small abnormalities a runner might have with a 2D video-based system, says Leitch.
‘You are talking about two or three degrees variation in the rotation of a joint,’ she says, ‘which will only start to become a problem when you run 40 miles a week.’
