I'm not sure about ESTER but for SOFR, the FED has stated that the way to get forwards is derive them from SOFR futures.


See section A.2 in this paper:


The answer looks to be similar for ESTER as well, the hope is by the decommissioning of LIBOR, swap and or futures markets will be liquid enough.