A novel colour-based method to detect road signs directly from videos is presented. A road sign is usually painted with different colours to show its functionalities. To detect it, different detectors should be designed to deal with its colour changes. A statistic linear model of colour change space that makes road sign colours be more compact and thus sufficiently concentrated on a smaller area is presented. On this model, only one detector is needed to detect different road signs even though their colours are different. The model is global and can be used to detect any new road signs. The colour model is invariant to different perspective effects and occlusions. After that, a radial basis function (RBF) network is then used to train a classifier to find all possible road sign candidates from road scenes. Furthermore, a verification process is applied to verify each candidate using its contour feature. After verification, a rectification process is used for rectifying each skewed road sign so that its embedded texts can be well segmented and recognised. Due to the filtering effect of the proposed colour model, different road signs can be very efficiently and effectively detected from videos. Experimental results have proved that the proposed method is robust, accurate and powerful in road sign detection.