Its supposed to work as it does. Each point can be used to promote or demote the score of a post/reply. You can use several on one post (eg bring its rating up from 2-5) or you could use them to add/remove 1 from the scores of several posts.
This way its averaged out. One unfair person cant do much damage, and the rating reflects usually the oppinions of several people.
See
www.slashdot.com and look at their comment pages for news articles. They have a similar system which inspired this one and it seems to work very well.
Of course when its finished you will be able to 'earn' more points. Example 1 point for every three posts you make, and an additional bonus point for every few times your posts get promoted.
Edit: Gave everyone 25 ratepoints for now, while I am finalizing the system.