A convolutional neural network (CNN)-based approach for fetal ECG detection from the abdominal ECG recording is proposed. The flow contains a pre-processing phase and a classification phase. In the pre-processing phase, short-time Fourier transform is applied to obtain the spectrogram, which is sent to 2D CNN for classification. The classified results from multiple channels are then fused and high detection accuracy up to 95.2% is achieved and the CNN-based approach outperforms the conventional algorithm. The hardware of this fetal ECG detector composed of the spectrogram processor and 2D CNN classifier is then implemented on the FPGA platform. Because the two dimensions of the spectrogram and the kernel are asymmetric, a pre-fetch mechanism is designed to eliminate the long latency resulted from data buffering for large-size convolution. From the implementation results, it takes 20258 clock cycles for inference and almost 50% computation cycles are reduced. The power consumption is 12.33mW at 324KHz and 1V for real-time operations. The implementation demonstrates the feasibility of real-time applications in wearable devices.