Hand paralysis is one of the most common complications in stroke patients, which severely impacts their daily lives. This article presents a wearable hand rehabilitation system that supports both mirror therapy and task-oriented therapy. A pair of gloves, i.e., a sensory glove and a motor glove, was designed and fabricated with a soft, flexible material, providing greater comfort and safety than conventional rigid rehabilitation devices. The sensory glove worn on the nonaffected hand, which contains the force and flex sensors, is used to measure the gripping force and bending angle of each finger joint for motion detection. The motor glove, driven by micromotors, provides the affected hand with assisted driving force to perform training tasks. Machine learning is employed to recognize the gestures from the sensory glove and to facilitate the rehabilitation tasks for the affected hand. The proposed system offers 16 kinds of finger gestures with an accuracy of 93.32%, allowing patients to conduct mirror therapy using fine-grained gestures for training a single finger and multiple fingers in coordination. A more sophisticated task-oriented rehabilitation with mirror therapy is also presented, which offers six types of training tasks with an average accuracy of 89.4% in real time.