A good advice when it comes to the Hull-White model is to never work with the short rate $r(t)$ directly. It will typically be quite unstable and depend on interpolation on the yield curve. Instead introduce the new variable $$ x(t) = r(t) - f(0,t), $$ where $f(0,t)$ is the instantaneous forward rate at time $t$ seen from time $0$. By rewriting the Hull-White model in this variable, the only quantities you will need from the yield curve to price the bond $P(t,T)$ are the market discount factor to $t$ and $T$.
Here, the book "Interest Rate Modeling" volume II by L. Andersen and V. Piterbarg is an excellent reference.