|
Slide 20 -
|
Information Extraction from Cricket Videos Syed Ahsan Ishtiaque
Kumar Srijan
Problems Statement There can be many challenges like face detection and recognition, boundary detection, separating out the ad and the cricket part and many more.
The problems we addressed are
Shot Transition Detection
Crowd Detection
Pitch Detection
Bowling Side Detection
Ball by Ball segmentation
Summary Making Shot Transition Detection A shot is defined as a sequence of frames captured by the camera in a contiguous way, without interruptions.
Shots are of two types
Hard Transition
Soft Transition
Following are our solution to this problem
Normalized Cross Correlation Threshold the correlation between every 20th frames
By checking the correlation between every 20th frame, and if its greater than certain threshold assuming that some transition has occurred, then checking for correlation between every frame in that interval, if any correlation value is greater than another threshold, then Hard Transition else Soft Transition
Results and Inferences
The results were satisfactory for Hard Transition, but got false matches for soft transition in the case when there was actually no transition and the camera was moving very fast, as the correlation value for the Kth and K+20th frame crossed the threshold.
Also there was a problem that the correlation between the frames of crowd was generally low, so generally every frame of crowd was classified as a shot transition
Histogram Based Histogram based Detection
By calculating the Histogram for all the bands or for gray band, and thresholding for sharp change of values we can detect for a transition.
Results and Inferences
This gave good results, the problem with this approach was that it was not taking the position of pixels into account
Every frame here is a hard transition, but the Histogram approach will not be able to detect this Difference of NCC Difference of correlation values between consecutive frames
In this approach we checked the difference of correlation, and if it is greater than a certain threshold we declare it as a hard transition.
Also blurring of the frames improved correlation and hence the results
Results and Inferences
This also gave very good results, and by this approach the problem of classification of crowd frames generally as shot was also solved, as although the correlation between the frames of crowd was low, but difference of correlation between the two low correlation frame groups was not that high, so it solved the problem. Crowd Detection Detecting the scenes in which crowd is present or is in focus. Histogram Based Histogram based
In a cricket video, we usually get two kinds of frames, one with field where there are very narrow range of colors present and of crowd where many different kinds of colors are present.
The histogram of crowd will be flat and will cover the whole range, whereas other scenes will have histograms which will be concentrated in a narrow range.
Results and Inferences
The results were not that encouraging. A single band or combination of bands, and also the range of values within a histogram could not be determined for histogram to be constructed and analyzed.
Edge Based Detection Edge Based
We observed that energy profile of the frame containing crowd is distinctly higher than those of not containing crowd, so we used canny edge detector and build a edge map of all the frames, now comparing the energy profiles of the edge map solved our problem
Results and Inferences
The results were very good Pitch Detection Finding whether the frame is showing a pitch or not and determining its position in the frame. Template Based solution Template Matching
We took a narrow horizontal strip, from the middle of a frame showing the pitch, as the template. For each frame in the video, we looked whether the strip can match somewhere near the middle of the frame. We used normalized correlation and square difference error as the matching standards.
Results and Inferences
The results were not that good, had few false positives in the frame showing the field.
The problem with template matching was that it did not address the variation of color of the pitch under different lighting conditions. HSI space based As template matching didn’t look into the variation of lighting conditions
The idea is that any frame not having the pitch will not be able to match in all the planes, viz. Hue, Saturation and Intensity, even if we take sufficiently large margins so that all the frames showing the pitch are correctly classified
For each frame, we broke it into HSI space and applied thresholding on all the planes according to the ranges calculated before.
We dilated each of the planes for filling the small holes and then took the intersection of the resulting planes.
We again dilated and eroded the resulting plane to remove the holes.
After that we took the distance transform of the image and searched for the maximum value in that. Thresholding this maximum value worked as the blob detection Results The results were very good Intensity Saturation Hue Intersection Blob Original Bowling Side Detection To detect from which side the bowler bowled.
Solution
Erosion of the blob obtained above shows the skeleton of the pitch which is available.
Whenever a bowler bowls the skeleton becomes “L” shaped due to the occlusion caused by the bowler.
By detecting the orientation of “L” we can easily determine the side from which the ball was being bowled Results This method gave very good results Hue Intersection Intensity Skeleton Original Saturation Difficulties Later in the video as the part of the pitch was covered with shadows, the “L” shape got truncated and it was hard to determine the bowling side. Skeleton Original Truncated “L” Bowling side not detected, but pitch was still detected Ball by Ball Segmentation To segment the video into deliveries
Solution
The detection of “L” can also help in knowing when the ball is being bowled. When the skeleton of the pitch changes from rectangle to L shaped then a ball was bowled. We also made sure that there was a gap of a few seconds between the deliveries by setting a timer. Results This was not classified as a delivery due to the timer Classified as a delivery Again Classified as delivery, as timer got reset Summary Making To generate a meaningful summary out of the video.
Solution
By a simple assumption that crowd is displayed only when there is six, four, wicket or sometimes ads.
So, every time crowd is detected we can go back few seconds and take that part as highlight.
Now sometimes crowd is displayed after an ad break, so now pitch detection helps here, every time we go back we can check for the presence of pitch, if it is detected then it is not from the ad break and we can include that part in our summary and if it is not we can skip that.
|