Electrochemical discharge machining (ECDM) is demonstrated to be a potential process for 3D microstructuring of Pyrex glass. However, the key to widening ECDM micromilling applications lies in how to improve the machining accuracy. To improve the machining quality of the ECDM micromilling process, microgroove machining experiments were conducted in this study. Three factors affecting ECDM micromilling performance - pulse voltage, tool rotational rate and travel rate of tool - were taken up as machining parameters to investigate their influences on machining performance. The results indicate that optimum combinations of both pulse voltage and tool rotational rate will realize better machining accuracy. The feasibility of three-dimensional microstructure machining was demonstrated by layer-by-layer ECDM micromilling machining. Complex structures were made to demonstrate the great potential for the 3D microstructuring of Pyrex glass of the ECDM micromilling process.