Now it is possible. They added "Manage Releases" tab to Google Play Developer Console. From Artifact Library under the Release Management tab, you can download your app's APK.

You need to have the necessary permissions to see the download button next to your artifacts. If in doubt, try the root account.