In the previous study by Huang and Lee (arXiv:1004. 1753) we introduced the well-posed boundary conditions P -,L0 and P +,L1 for the odd signature operator to define the refined analytic torsion on a compact manifold with boundary. In this paper we discuss the gluing formula of the refined analytic torsion for an acyclic Hermitian connection with respect to the boundary conditions P -,L0 and P +,L1. In this case the refined analytic torsion consists of the Ray-Singer analytic torsion, the eta invariant and the values of the zeta functions at zero. We first compare the Ray-Singer analytic torsion and eta invariant subject to the boundary condition P -,L0 or P +,L1 with the Ray-Singer analytic torsion subject to the relative (or absolute) boundary condition and eta invariant subject to the APS boundary condition on a compact manifold with boundary. Using these results together with the well known gluing formula of the Ray-Singer analytic torsion subject to the relative and absolute boundary conditions and eta invariant subject to the APS boundary condition, we obtain the main result.