[AVTCORE] MEXaction2 action detection and localization dataset available

[Bernard Merialdo <[e-mail]>]

(In behalf of Michel Crucianu and Jenny Benois-Pineau)


We are happy to make public the MEXaction2 action detection and 
localization dataset. A detailed description, including video samples, 
the evaluation procedure and baseline results, is available at
http://mexculture.cnam.fr/xwiki/bin/view/Datasets/Mex+action+dataset

A brief description follows.

The aim of the MEXaction2 dataset is to support the development and 
evaluation of methods for 'spotting' instances of short actions in a 
relatively large video database. For each action class, such a method 
should detect instances of this class in the video database and output 
the temporal boundaries of these detections, with an associated 
'confidence' score. This task can also be seen as 'action retrieval': 
the 'query' is an action class and the results are instances of the 
class, ordered by decreasing 'confidence' score.

The dataset contains videos from three sources:

  1. INA videos. A large collection of 117 videos (for a total of 77 
hours), was extracted from the archives of the Institut National de 
l'Audiovisuel (France). It contains videos produced between 1945 and 
2011.  The video content in this collection was divided into three 
parts: training, parameter validation and testing.

  2. YouTube clips. From additional videos collected from YouTube, we 
provide 588 short clips, each containing only one instance of an action 
to spot (everything within the boundaries of a clip belongs to an action 
instance). All these instances should be only used for training.

  3. UCF101 Horse Riding clips. We add as training instances for the 
HorseRiding class the Horse Riding clips from the UCF101 dataset. All 
these instances should be only used for training.

There are two annotated actions (see the abovementioned website for 
examples):

  1. BullChargeCape: in the context of a bull fight, the bull charges 
the matador who dangles a cape to distract the animal.

  2. HorseRiding: instances of one or several persons riding horses.

The numbers of annotated examples for the two actions are:
  BullChargeCape   1324
  HorseRiding       651

Beside the fact that the total amount of annotated video is relatively 
large compared to other existing datasets, this dataset is also 
interesting because it raises several difficulties:

  1. High imbalance between non-relevant video sequences and relevant 
ones (instances of an action of interest).

  2. High variability in point of view and background movement (and 
action duration for HorseRiding).

  3. Variability in image quality: old videos have lower resolution and 
are in black and white, while the newest ones are in HD.

Action detection and localization is evaluated as a retrieval problem: 
the system must produce a list of detections (temporal boundaries) with 
positive scores. Sorting these results by decreasing score allows to 
obtain precision/recall curves and to compute the Average Precision (AP) 
in order to characterize the detection performance.