Since all that we're keeping track of is the order that balls are thrown and caught, what we mainly notice is when two balls are thrown in a certain order but caught in the opposite order.
In general this gives one a process from which to construct new patterns from old: pick two throws such that the earlier throw lands after the later throw is thrown - throw, throw, catch, catch rather than throw, catch, throw, catch. Fiddle with the heights so those two balls go to one another's place, "swap" their "sites", and you have a new pattern. (The assumption that the first land after the second is thrown is what prevents the new pattern from having a ball caught before it is thrown, which would be rather strange!)
This is easy to do to a siteswap numerically. If the throws are h apart, to heights a and b in that order, make them instead be to heights b+h, a-h.
So to prove that the average of the numbers is the number of balls, it will be enough to show that we can site swap any pattern down to the constant pattern N, N, N, N which has average N and obviously has N balls.
The process is this. Find a number in the pattern that is larger than the number after it ("after" taken in a wraparound sense - in the pattern 12345, for instance, only the 5 has a number after it that is smaller, namely the 1). They must differ by a least 2, or else there would be a collision (and we've disallowed multiplexing). So when we swap these two sites, the pattern is evened out some. After a finite number of steps all the numbers become equal (the only time when there is no number with a smaller one following). QED.
On the 12345 example: here is a chain of siteswaps flattening this pattern to 3: 12345 -> 42342 -> 33342 -> 33333. For a worse example, take 801 -> 171 -> 126 -> 522 -> 342 -> 333.