Main results achieved during Year 1 (Feb. 2010 - Jan. 2011)


Automatic sensor selection for videowall management

During the first year, IDIAP and MULT has studied different models for unsupervised activity modelling allowing to automatically select the most relevant/salient video streams to be displayed on videowall in control rooms. The main achievements were:

      • several models that are able to discover recurring and co-occurring activities in video clips from long term observations (learning stage),
        that also explicit model temporal information (see publications BMVC, CVPR), to discover temporal relationship between activities such as cycle (see ICIP);
      • several anomaly indices allowing to identify unusual activity pattern in single or multiple camera data streams (see AVSS-1, AVSS-2); and
      • applications to single or multi-stream video data from the Turin metro, with encouraging preliminary results on on-the-fly activity recognition and unusual behaviour detection.

Additionally, during this first year, TCF has worked on two main directions for audio surveillance. The first one was to study and develop abnormal audio event detection algorithms (see publication SSP); the second was to study several issues relating to semantic analysis of audio signals for long term behaviour analysis and automatic audio stream selection in control rooms.


Human-centered monitoring using audio/video analysis

Individual level: Body pose estimation


During the first year, IDIAP has worked on a Multi-Object-Tracking algorithm, using tracklets reliably following individual people. The second main work was to study several issues relating to the estimation of the behavioural cues, for head localization and body orientation estimation. In that task, IDIAP has developed a framework for body orientation estimation relying on a novel body pose classifier. Results on videos from Turin, the Caviar surveillance database, and indoor cameras demonstrate the good performance of the approach.

Group level: Group detection and tracking


For the task of group interaction and group tracking, a first prototype has been developed by INRIA. This prototype is based on an individual tracker’s output. The decision that tracked people are interacting together in a group is based on their distance, their speed and direction similitude. A time window delays the decision of grouping people for a greater robustness to individual tracking failures. The developed algorithm is able to keep track of groups for a long period of time, even if individual members move temporarily away from others. Concrete evaluations is planned for year #2 using annotated data (ground truth).

Crowd/Flow level: People flow monitoring in escalator


Regarding the people flow/crowd monitoring, MULT evaluated one algorithm providing a continuous measurement of people flow in escalator. The preliminary evaluation results shows relatively good error rate on cumulative counting. Some qualitative experiments also show that it is then possible to identify different weekday and week-end trends related to the escalator usage by analysing one week of data. Next steps are related to an exhaustive performance evaluation on more/different camera views to explore the links between consecutive escalators, to measure the domino effect related to exiting flows.

Data collection and behavior annotation


During this first year, GTT, together with THALIT, has set-up the system to perform the data acquisition in Turin trial site. Following this acquisition system set-up, an audio-visual dataset has been acquired at GTT and shared with the consortium. In the meanwhile, several hours of video material were annotated by UNIVIE following the behavior catalogue, i.e. all behaviours of all people visible on the current video material.


Long-term statistics building for planning applications

Not started yet; first results available around April 2012.