Will cancelling credit cards affect credit score?

Talk to the credit card issuers and explain your dilemma. Explain that you are worried that your CC numbers might have been compromised. They might be willing to issue you a new card with new numbers. Then you don't have to worry about the cancelation showing on your credit reports.