Yes you do have to be careful what you use on marble. Many times water alone is not sufficient. There are many cleaners available for use on calcite stones like marble, limestone, travertine. This Old Grout Stone Prep is an excellent cleaner for soiled marble and is safe. Stone Care International makes some very good products also. MBstonecare has some great products.
Using just water will pick up some soil, but you are really just moving the soil around on the surface. Modernstonecare.com has a variety of products for use on stone.