To reproduce the duration of an event precisely, one needs to represent the temporal information without being influenced by other magnitude attributes (e.g., size) of the event. In the present study, however, task-irrelevant numerical magnitude automatically affected participants' reproduction of the duration of a stimulus. In Experiment 1, participants made key-press responses to reproduce the duration of numbers. Reproduced durations were shorter for small numbers (e.g., 1) than for large numbers (e.g., 9). In contrast, in Experiment 2, participants' reproductions of a standard duration were longer when their key-press response was accompanied by visual presentation of a small number than when it was accompanied by presentation of a large number. These results clearly demonstrate that number-time interference extends beyond simple mapping between stimulus categories and response alternatives. The findings support the notion that either a common magnitude representation or closely connected magnitude representations underlie numerical and temporal processing.