Owing to many appealing properties, neural networks provide a natural basis for solving different kinds of problems. The performance of neural networks greatly depends on whether they can provide appealing solutions to the problems of the parameter learning (i.e., the connecting weights in each layer) and the structure learning (i.e., the network structure). These two kinds of learning can be performed simultaneously or separately. In this paper, we proposed the Jacobian matrix-based learning machine (JMLM) to provide an appealing solution to the aforementioned two kinds of learning. The network structure of a JMLM can be incrementally constructed and a Jacobian-matrix-based learning method is proposed to efficiently estimate the corresponding network parameters. Furthermore, we can provide physically meaningful explanations to help human analyzers to make decisions based on the parameters embedded in a trained JMLM. One 2-D artificial data set, one benchmark medical data set, and an intensive care unit survival prediction data set were used for demonstrating the performance of the proposed JMLM.