Use setRoundingMode, set the RoundingMode explicitly to handle your issue with the half-even round, then use the format pattern for your required output.

Example:

DecimalFormat df = new DecimalFormat("#.####");
df.setRoundingMode(RoundingMode.CEILING);
for (Number n : Arrays.asList(12, 123.12345, 0.23, 0.1, 2341234.212431324)) {
    Double d = n.doubleValue();
    System.out.println(df.format(d));
}

gives the output:

12
123.1235
0.23
0.1
2341234.2125

EDIT : The original answer does not address the accuracy of the double values. That is fine if you don't care much whether it rounds up or down. But if you want accurate rounding, then you need to take the expected accuracy of the values into account. Floating point values have a binary representation internally. That means that a value like 2.7735 does not actually have that exact value internally. It can be slightly larger or slightly smaller. If the internal value is slightly smaller, then it will not round up to 2.7740. To remedy that situation, you need to be aware of the accuracy of the values that you are working with, and add or subtract that value before rounding. For example, when you know that your values are accurate up to 6 digits, then to round half-way values up, add that accuracy to the value:

Double d = n.doubleValue() + 1e-6;

To round down, subtract the accuracy.