We can't apply the custom formatting within the Dossier like we can in a report. However, we can apply custom formatting in a report and convert it into a Dossier. This workaround will carry forward the custom column formatting into the Dossier!
Here is my report with custom Column headers.
When I'm ready to put this into a Dossier I can select Tools ➡ Create Dossier.
Now my custom formatting has been carried forward!
The downside is that none of the custom formatting can be modified from within the Dossier. This means that if I want to make a change, I'll need to perform this process again.
And that's it!