The uncertainty involved in the interpreted geological model may be categorized as the stratigraphic uncertainty and the properties uncertainty. Note that although the influence of the properties uncertainty on the behavior or performance of the geotechnical system and the geotechnical design has been extensively reported in the literature, the studies that address the stratigraphic uncertainty are limited. This paper presents a study regarding the influence of the stratigraphic uncertainty on the behavior of the geotechnical system and the geotechnical design. In which, the uncertainty in the stratigraphic configuration is characterized using the stochastic Markov random field-based approach with a large number of potential stratigraphic realizations. With these stratigraphic realizations as inputs, the influence of the stratigraphic uncertainty on the behavior of the geotechnical system is evaluated in a probabilistic manner; then, the design of the geotechnical system is formulated as a bi-objective optimization-based problem that considers the design safety and the cost simultaneously. To demonstrate the effectiveness of the proposed probabilistic analysis and design approach, the problem of designing stabilizing piles in a slope consisting of multiple strata is studied. The parametric study is further conducted to analyze how the probabilistic analysis results are influenced by the pile parameters and how the probabilistic design results are influenced by the additional boreholes.