In this paper, a greedy and A*-based searching algorithm is proposed to find the optimal morphological filter on binary images. According to the Matheron representation, the estimator for mean square error (MSE) is defined as the union of multiple erosions. Unfortunately, finding the optimal solution is a long search and time consuming procedure because we have to compute the MSE values over all possible structuring element combinations and make comparisons among them. In this presented paper, the search for the solution is reduced to the problem of obtaining a path with the minimal cost from the root node to one vertex on error code graph. Two graph searching techniques, greedy and A* algorithms are applied to avoid the search on the extremely large number of search space. Experimental results are illustrated to show the efficiency and performances of our proposed method.