How should spackled and sanded drywall be prepared for primer and paint?

When the spackling and sanding is finished, should the wall be wiped down with a dry rag or a damp rag or sponge to get rid of the fine dust. Will a damp sponge ruin the sanded spackling?