If you do try to clean them, soap and water sounds good, just make sure to let them dry well.

But I would recommend just replacing them, as you will have to take them off to clean anyway. Also headphone ear cushions are rather inexpensive, and will make your headphones feel like they are brand new again.