A technical memo that I came across by Alvy Ray Smith from July 15, 1995. A good point is made here. Every other book/article/graphics tutorial always models a pixel as a square.

A pixel is a point sample. It exists only at a point. For a color picture, a pixel might actually contain three samples, one for each primary color contributing to the picture at the sampling point. We can still think of this as a point sample of a color. But we cannot think of a pixel as a square—or anything other than a point. There are cases where the contributions to a pixel can be modeled, in a low-order way, by a little square, but not ever the pixel itself.