Hypervideo is a new viewing way for films just as hypertext on the WWW. In the literature, many standards and technologies have been developed for the hypertext. However, the present video processing methods cannot provide a proper solution for hyperlinks in video. In this paper, we propose an object-based hypervideo authoring system. Video objects can be described by semantic annotation and multistory movies can be produced. The system offers a manual object-based interface for the film producer to select meaningful video objects and annotate them. It also has a video construction function to facilitate the audience to view the annotation of interesting objects in a scene or to select a particular viewing path. It is very useful to show the annotation of a particular object that the film producer wants to highlight in educational videos or commercials. A hypervideo player is also developed to play the video file made by the authoring system.