I am also using very hard tap water (GH from 21 to 26, and KH around 12) and I can manage to have good growth without relevant algae problem (obviously not completely algae fee tank, if this exists...). Not really sure if a softer water would make my life easier concerning CO2 management (especially consuming much less co2) and some some nutrients uptake... but I really doubt this is your problem. My low tech nano (also using this water) is even more algae free...
I suffered from brown diatoms in my nano tank some months ago and I tried everything written and explained in this and other forums... But I finally I understood that a complete reboot was needed. I cannot explain why: too much algae spores on the substrate? Bad condition of the plants remaining? My guess is that too many things were wrong and that without experience it was going to be difficult to reboot a tank without a major change. This is why I got rid of all the plants that were not perfect, I took out all the equipment and hardscape and cleaned it very well. Then I vacuumed the substrate (something I don't do anymore), went to the LFS and grabbed some bunches of plants. I also added all the easy plants I had from the other tank and friends... And began from scratch. I'm subscribed to your thread and honestly I cannot imagine what is going wrong. It looks that you have work on your flow, co2 profile, light...
Jordi
Edit: I haven't found what's your KH. What is your dropchecker saying. Another thing I have notice in my hard water tanks is that I need a yellow DC at lights on (fish and shrimp are perfect), if my DC is just lime green I have BBA (I'm pumping 2 bps of co2 during 3 hours before lights on! Otherwise I don't get the levels I need to avoid BBA)