Brown diatoms is tank balance related, meaning it's excess ammonia/bioload not being consumed on time by: filter bacteria and or plants.
I get it when I : add too many fish at a time, disturb the substrate, consistently overfeed. It goes away in 2-3 months or less as the tank starts coping consistently with the bioload I've added. I used to raise guppy fry a lot before, the tanks were in a constant state of diatoms. I fed them a lot and I fed them all sorts of tank polluting stuff like boiled eggs, etc.
So all the reasons mentioned here are possible cause. For some it maybe CO2 as it will impact plant health which in turn can cause elevated ammonia for one or another reason as the plants are either producing organics, or not taking up ammonia and this will last until the plant issue is solved, more fast growers are added or the bacteria in the filter starts multiplying to cope with the extra bio they've been given all of a sudden.
Another thing I noticed is you throw some of your filter sponges away and replace with new. I'd start cleaning them instead and return to the filter as that could cause the minor elevated ammonia enough for diatoms to thrive. Over cleaning a filter is more likely to cause diatoms than not cleaning it regularly. It's best to clean the tank thoroughly, not the filter.
Christos mentioned purigen and that can help too as it takes up organics, thus reducing the potential ammonia production. Ottos of course love the stuff.
There are a ton of other reasons why there could be elevated ammonia. Oxygen content, etc.. One should use common sense and think what that could be in their own setup.
Edit: When I say "elevated ammonia" I don't mean an amount that a home test will detect. It's tiny amounts more than what is normal for a healthy tank. The last time I got diatoms was in two tanks simultaneously after I took all plants and shrimp from one of the tanks(100 or so shrimp I stopped counting) and moved all the shrimp to the other tank. The plants went into a bucket. The bare tank with all plants removed got a diatom outbreak. The other tank to which I added all shrimp in one day got a diatom outbreak. I tested only the tank I left bare because I still had a small fry fish in it and I was worried and the ammonia was reading 0.25ppm which I don't normally see at all in normal scenarios. But this lasted just a day where the diatoms lasted a couple of months.
Both tanks had been running for years so well established.