Motion of a point charge emitting radiation in an electromagnetic field obeys the Abraham-Lorentz-Dirac equation, with the effects of radiation reaction or self-force incorporated. This class of equations describing backreaction, including also the equations for gravitational self-force or Einstein's equation for cosmology driven by trace anomaly, contains third-order derivative terms. They are known to have pathologies like the possession of runaway solutions, causality violation in preacceleration, and the need for an extra second-order derivative initial condition. In our current program we reexamine this old problem from a different perspective, that of non-Markovian dynamics in open systems. This conceptual and technical framework has been applied earlier to the study of backreaction of quantum field effects on charge and mass motions and in early universe cosmology. Here we consider a moving atom whose internal degrees of freedom, modeled by a harmonic oscillator, are coupled to a scalar field in the same manner as in scalar electrodynamics. Due to the way it is coupled to the charged particle, the field acts effectively like a supra-Ohmic environment, although the field itself actually has an Ohmic spectral density. We thus have cast the problem of radiation reaction to a study of the non-Markovian dynamics of a Brownian oscillator in a supra-Ohmic environment. Our analysis shows that (a) there is no need for specifying a second derivative for the initial condition, and (b) there is no preacceleration. These undesirable features in conventional treatments arise from an inconsistent Markovian assumption: these equations were regarded as Markovian ab initio, not as a limit of the backreaction-imbued non-Markovian equation of motion. If one starts with the full non-Markovian dynamical equation and takes the proper Markovian limit judiciously, no harms are done. Finally, c) there is no causal relation between the higher-derivative term in the equation of motion and the existence of runaway solutions. The runaway behavior is a consequence that the memory time in the environment is shorter than a critical value, which in the case of the charged particle is the classical charge radius. If the charge has an effective size greater than this critical value, its dynamics is stable. When this reasonable condition is met, radiation reaction understood and treated correctly in the non-Ohmic non-Markovian dynamics still obeys a third-order derivative equation, but it does not require a second derivative initial condition, and there is no preacceleration.