If you plan on inviting a big group of people over for an afternoon or evening of swimming and having fun, one thing that will be important is that you make sure everyone will have food; this does not mean that you have to provide food for everyone, but it does mean that you should have a grill ready to go throughout the day, and you should let people know that they should bring their own meat if you will not be providing any for them!
In order to set the atmosphere of "fun" for the day, it will be important that you have music playing outside, and you will also want to have the pool set up in such a way that any of a variety of pool games can be played.
And finally, the biggest key of all to throwing a successful pool party is inviting the right people; when you have a good group of friends over - friends who know how to have fun together - you can have a great pool party, no matter how all the other aspects of the day happen to go!