No, you do not need to use different spindles.  The main factor will be how fast the feed rate will be that you run and how much material you remove per pass.

If you with to run a light duty spindle like a Dremel 4000, you can run wood, plastics and metals just as well as if you were using a Kress 800W spindle. The only difference is that you will not be able to remove as much material and your feed rate will be slower the harder the material is.

Keep in mind that dedicated CNC spindles will have better quality bearings which will offer less runout and therefore a longer life span and higher accuracy.  Using the Dremel example, this device is not meant for hours and hours of prolonged use, nor is it made to have lots of pressure side loading the main shaft.  You will find that if you are not careful on the feed rate and pass depth that you will quickly wear out the bearings.

A router like the Dewalt DW611 will also cut all types of material, but as with the Dremel, the bearings are not as good as with a dedicated spindle so you can expect more runout with a router.  The STEPCRAFT HF500 is a purpose built spindle that was designed to work with all types of materials and has an excellent bearing system that provides very little runout and a much higher accuracy.

The bottom line is that you can pretty much cut whatever you like with just about any spindle.  You just need to be cautious on feed rates and pass depth.