I've finally had some success with creating my own detail maps, so I thought I'd share what I discovered.
Step 1. Find a good image to make your detail map from.
Step 2. Make a seamless texture out of the image (see my 'creating seamless textures' tutorial here) and resize it to 256x256 pixels.
Step 3. Create an alpha channel and put some texture into it - note, it's a bad idea to just use a desaturated version of the color image as the alpha channel. You should use a different texture, or at least offset the alpha channel slightly. The alpha channel should be seamless too of course. The black part of the alpha channel is transparent and the texture stamping will show through this.
Step 4. Now save your image as a .dds file (you can get the .dds plugin for Photoshp at Nvidia's website). Be sure to save the .dds with the following settings:
Note you can also use DXT5 compression - this may give a smoother alpha channel. Experiment to see what works best for your particular texture.
If you don't need the texture stamping to show through the detail map (say if it's a rock texture for example) then you could save it as DXT1, with no alpha. Save the image to this folder:
Dawn of War\My_Mod\Data\Art\scenarios\textures\detail
replacing My_Mod with the appropriate folder name.
You will now see the texture stamp show through your detail map's alpha channel.
If you've saved your map here:
\Dawn of War\My_Mod\Data\Scenarios\MP\mapname.sgb
then the 'mapname.tga' file will be the start of your texture stamping image (thanks to Rymafyr for working this one out). This is the image that shows through the gaps in your detail map. You can paint it directly in photoshop or you can paint it with the Mission Editor's texture stamping tools. It's a good idea to paint this image to match the colors of your level, as it has a very large impact on the icon and minimap images which are generated by the Mission Editor whenever you save your map.
Good luck with your levels!