Found this fix on another forum. Click the quote to make it expand.
1) I put the phone in airplane mode and deleted ALL the messages in my voicemail. Then, I made sure that all of them, including the one zombie message that would keep coming back to life, was also in the deleted items foldder. If for some reason I could not delete the message by hitting the delete button, I would swipe the message field of the message I wanted to delete to the right and then press the red delete button that would appear.
2) With the phone still in airplane mode, I then went to the deleted voicemail folder and selected Clear All. If for some reason the zombie message won't disappear, use the swipe right technique to delete it.
3) Then, go back to the regular voicemail foler to ensure it is totally clear of all messages. Then doublecheck the deleted voicemail folder and make sure it also is totally empty. You may not be able to get to the deleted voicemail folder if it is totally empty but that is the quality check that it truly is empty.
4) Then, with the phone still in Airplane mode, shut off the phone by pressing the on/off button on the top right hand side and then turn off the phone.
5) When the phone is shut down, turn it on again.
6) Make sure that when the phone turns back on that it still is in Airplane mode. Double check the voicmail folder to ensure that it is still empty. Now, do a reset of the phone by holding down the on/off button and the home key until the Apple logo appears on the screen.
7) When the phone powers back up, wait until you see a SOLID red dot above the voicemail icon. Once that appears, you are good to go.
8) Go back to settings and turn off Airplane mode.
9) This should fix the issue as it worked for me
Hope this works for you too and GOOD LUCK!
I followed those steps, but didn't see a red dot above the voicemail icon. Should I repeat the steps? All of the voicemails are deleted. I'll have to test it out and leave a new voice mail to see if the old ones appear.