Falling activity is a critical behavior due to the physical discomfort for elders. The prime time of rescuing is missed whenever falls accidentally happen. Fall detection in real time could save human life in video surveillance systems. Recently, digital cameras are installed everywhere. Human activities are monitored from cameras by intelligent programs. An alarm is sent to the administrator when an abnormal event occurs. In this paper, a multi-view-based manifold learning algorithm is proposed for detecting the falling events. This algorithm should be able to detect people falling down in any direction. First, the walking patterns in a normal speed are modeled by the locality preserving projection (LPP). Since the duration of falling activity is hard to be estimated from real videos, partial temporal windows are matched with the normal walking patterns. The Hausdorff distances are calculated to estimate the similarity. In the experiments, the falling events are effectively detected by the proposed method.