The format for the function you refer to is:
ROUNDTONUMBER (NUMBER, ROUNDTO) or
ROUNDTONUMBER (NUMBER, ROUNDTO, #DECIMALS)
Rounds NUMBER to the nearest ROUNDTO value, such as to the nearest 10, 25, 50, 100, 250, 500, 1000, 5000, 10000, etc.
In your example, I would use:
[%ROUNDTONUMBER((((WPSM5Price+WPSM6Price+WPSM7Price+WPSM8Price)/4)*.7)+ HowManyRoomsHouse,500)%]