Why X'X inverse times X'Y?

If you ever wind up calculating multiple linear regression yourself, you'll undoubtedly run into (X^TX)^-1X^TY, a cryptic looking expression which might be mistaken for a magic word from an old adventure game.

What's the story behind the magical expression(X^TX)^-1X^TY? What is it? How did it get that way? These are some questions I'll take a stab at answering in upcoming posts.

Next: Dot products of orthogonal vectors.