Odds are the ball-IT is off center slightly. Like some have said, this causes the slug to push to one side of the hole.
Very interesting someone said the outside diameter of the colored slugs are different. I've been saying that since I got this kit when it first came out. I have gone exclusively to clear slugs for my customers. But even in some cases, two slugs of the same color can have different outside diameters. This can be a pain in the a** when drilling additional slugs for someone, so I just make sure to check that the slug goes in and out smoothly on their ball before putting a hole in it.
Otherwise, you end up reaming out the sleeves to fit one slug, and then the other slug ends up having wiggle room.
Got off subject a little there, but hey, GOOD LUCK! The IT really is an incredible system once you know how to avoid the common and not so common problems.