I solved it by making a seperate account for uploads. Originally I was trying to do it using the same account but it would not work even with the permissions setup properly. I even checked to make sure that the sub-folders were not inheriting the permissions of the upper folders.
Anyways got it working now with two seperate accounts, one for uploads and one for downloads..
