Finding a set of suitable experts with minimized communication overhead to perform a complex task finds a wide spectrum of applications in industry, education, and other scenarios. This class of problems, widely formulated as forming a team of experts in social networks (i.e., team formation problem), is very challenging due to its NP-hardness and has attracted much research attention. Although various effective and elegant algorithms have been proposed to address this important problem, the methods are usually manually designed and handcrafted, which require considerable human efforts. In this article, we make our first attempt to automate the algorithm design with a machine learning-based approach, named reinforcement learning-based expert team identification (RELEXT). Moreover, we also propose two novel graph embedding methods to consider two important dimensions of the team formation problem, i.e., the skill and social dimensions. We evaluate the proposed approaches on multiple large-scale real datasets. The experimental results show that our proposed approaches outperform the other baselines in terms of solution quality and efficiency.