Wimbledon is set to begin next week, and with it come spectators cheering and players whacking a tennis ball over the net, hoping to earn a prestigious victory in pursuit of tennis’ grand slam. But for the fans, all the action can be hard to follow.
This year, that could change. IBM’s Research and iX (business design) teams have created a way to curate video highlights of Wimbledon based on artificial intelligence and visual content analysis. Called “Cognitive Highlights,” the system captures cheering, players’ reactions, and many other points of information to create short video clips.
The system was first deployed in April at The Masters Tournament, where it detected the live video stream for a number of factors, among which are commentary, commentator voice and TV graphics. Then, using artificial intelligence, the system would select moments from the competition and send them to video producers at The Masters, who could choose segments and edit them into a full highlight reel from an interactive dashboard.
At Wimbledon, the system will actually create one- to two-minute highlight videos from the various segments of footage. Instead of Wimbledon’s digital editors having to sort through highlight clips, they’ll have small reels recommended to them by IBM’s technology.
“The reason why computer processing is so helpful in these kinds of events is because the sheer volume of play that’s happening at these live events,” said John Smith, an IBM Fellow and manager of multimedia and vision at IBM’s T.J. Watson Research Center. “With golf, all the players on different holes, everything happening in parallel, and similarly for tennis, there’s a lot of play that goes beyond the main court.”
Get The Latest Sports Tech News In Your Inbox!
For Wimbledon, “cognitive highlights” can be especially useful, since well-known players will often compete on any of six side courts. Matches on those courts draw much less attention, but can produce equally highlight-worthy plays. IBM’s technology will allow for Wimbledon to publish highlights from every court that features action, according to an IBM article announcing the new feature.
In addition to the highlight videos — which are ranked on a number of statistical, auditory and visual factors to produce an “excitement score” — the system also uses data from the sideline statistician and sensors like the speedometer to create video overlays that describe the highlight, according to the post. Together, these components form the highlight packages that IBM sends to Wimbledon to publish across their digital avenues.
“The idea is just to enhance the job of the editors, so instead of having them looking at the whole video data, which is a lot, we just provide a subset,” said Rogerio Feris, a research manager at the Watson Research Center. “You have this ranked subset of data, and then they can choose how to create (a) highlight reel from the data.”
Smith and Feris each see a possible opening for the system in other areas. Feris mentioned soccer, which also features continuous action up and down the pitch, while Smith said he sees an opportunity for the system to deliver personalized highlight videos to different people: “This scaling of the production process is very difficult to do manually, but it’s something that could be a natural output when you have this ability for the computer to go through and recognize things at a finer granularity and pull the relevant clips together.”
But for now, the system will track every moment of the Wimbledon championships, over a 13-day period, and sort through the tournament to produce only the most interesting serves, points and celebrations of each match for fans to watch, share and relive.