To address the problem of forecasting solar irradiance for grid operators, the aim of this work is to automatically predict solar irradiance several minutes in advance. This work presents a solar irradiance prediction scheme that utilizes features extracted from all-sky images. To select a proper feature subset for prediction, various features are analyzed and compared. We propose to utilize the regression technique to predict clearness index and then to calculate the desired solar irradiance from the predicted clearness index. We validate the effectiveness of the proposed scheme using a challenging dataset collected at a coastal site. The experiments have shown that the designed clearness index prediction mechanism yields better prediction results than predicting solar irradiance directly. Also, irradiance prediction at 5. min in advance can be achieved with mean absolute error of around 22%. The results of this work could provide very useful information for grid operators to ensure greater efficiency of the renewable energy supply.