We propose a simple model of inter-bank lending or borrowing with the corresponding delayed feedback.The evolution of monetary reserve of N banks is described by coupled square-root diffusions driven byregular controls and controls with delay in the drifts satisfying the optimization for coupled objectivefunctions. The amount of each bank desiring lending to or borrowing from a central bank is evaluatedthrough the comparison of its monetary reserve and the ensemble average. Systemic risk is characterized bythe large number of defaults, the monetary reserve reaching the given default barrier in the fixed time horizon.The equilibrium is obtained using fully coupled forward and anticipated backward stochastic differentialequation. We observe that the lending or borrowing leading to the mean-reverting at ensemble averagecreates stability but systemic risk. However, through the delayed feedback, the system of interbank lending orborrowing becomes diverse yielding the reduction of possibility of systemic risk.