I don't have an iPhone but rather an touch. I checked music under settings and there is no iCloud switch. But I did switch off iCloud under the iCloud setting but to no avail. More suggestions would be appreciated. Thanks.
In iTunes on your computer, press the music icon in the upper left, then My Music on the left side of the controls in the center, then select Songs in the upper right. Check the boxes for Show Artwork and Always Show and sync your iPod.
If the album art still doesn't show up after that, unsync all of your music through iTunes and then resync it.