Polarized and unpolarized neutron-diffraction measurements have been taken to determine the magnetic structure of the Cu spins in the RBa2Cu3O6+x system as a function of temperature. Most of the results have been obtained on two (semiconducting) oxygen-deficient single crystals of composition NdBa2Cu3O6.1 and NdBa2Cu3O6.35. On cooling from the paramagnetic state, the Cu-O plane layers first order at a Néel temperature TN1 in a simple antiferromagnetic arrangement of spins, with TN1=430 K for x=0.1 and TN1=230 K for x=0.35, and with a saturated moment of 0.65B. At a lower temperature TN2 the Cu chain layer also orders, with TN2=80 and 10 K for x=0.1 and 0.35, respectively. Hence both transition temperatures decrease with increasing oxygen concentration in this range of x, and the qualitative behavior for these two samples is found to be identical. We have not, however, observed this low-T structure in polycrystalline samples of either YBa2Cu3O6+x or NdBa2Cu3O6+x, and some possible explanations for this difference in behavior are discussed. For the single crystals in the low-T phase the Cu spins in all three copper-oxygen layers are ordered in a simple antiferromagnetic arrangement, with the magnetic unit cell double the chemical unit cell along all three crystallographic directions. On the basis of the measured integrated intensities of over 40 magnetic reflections in both ordered phases, the results can be accounted for quantitatively with the assumption of a 3d magnetic form factor on the Cu ions, with no need for a significant moment on any of the oxygen ions. At low T, site-averaged moments of 0.35B and 0.8B are obtained in the chain and the plane layers, respectively. The fact that the ordered moment on the plane layers increases in the low-T phase is direct evidence that two-dimensional quantum fluctuations play an important role in the plane layers in the high-T ordered phase. The evolution of the low-T spin structure into the high-T structure involves a noncollinear configuration in which the spins rotate as a function of temperature in the plane layers, while the chain layer becomes thermally disordered. The temperature dependence of the structure indicates that competing interactions between the layers are important, and this is discussed in detail. Finally, exploratory measurements as a function of applied magnetic field indicate that the magnetic anisotropy in the tetragonal plane is much smaller than the exchange energies.