Microswimmers such as bacteria exhibit large speed fluctuation when exploring their living environment. Here, we show that the bacterium Escherichia coli with a wide range of length speeds up beyond its free-swimming speed when passing through narrow and short confinement. The speedup is observed in two modes: for short bacteria with L <20 μm, the maximum speed occurs when the cell body leaves the confinement, but a flagellar bundle is still confined. For longer bacteria (L ≥ 20 μm), the maximum speed occurs when the middle of the cell, where the maximum number of flagellar bundles locate, is confined. The two speed-up modes are explained by a vanishing body drag and an increased flagella drag—a universal property of an “ideal swimmer.” The spatial variance of speed can be quantitatively explained by a simple model based on the resistance matrix of a partially confined bacterium. The speed change depends on the distribution of motors, and the latter is confirmed by fluorescent imaging of flagellar hooks. By measuring the duration of slowdown and speedup, we find that the effective chemotaxis is biased in filamentous bacteria, which might benefit their survival. The experimental setup can be useful to study the motion of microswimmers near surfaces with different surface chemistry.