I had this same problem for a while and finally figured it out!
I had 'rm' and 'rmdir' in the '/bin/' folder, but neither file was set to be an "executable". In WinSCP, I had to navigate to these two files, right-click, choose "Properties", and then change their permissions to:
Octal: 0755
Earlier, I thought that I was supposed to change the permissions on the files and folders I was trying to delete. Doh!