All this talk about about summing colors and only
one reply even mentions color spaces?

It doesn't make physical sense to directly add
sRGB colors; if you don't take
gamma into account, the result will be too dark (see the attached gradient comparison).
The (mostly) correct way would be to normalize each channel to the range 0.0 - 1.0, raise it to 2.2, calculate the average, undo the raising (e.g. by using 1 / 2.2 ≈ 0.4545 as the exponent) and finally scale back to 0 - 255. Much more computationally expensive of course, so you'll need to decide if the quality improvement is worth it.
TL;DR: Please linearize your colors before doing math on them
