In order to avoid looking like a tag, you need a way to specify < and > without using those actual characters.
But since & is used to specify a character reference, you need one for & as well which in this case is: &.
Similarly, you may need to use a quotation mark inside an quoted attribute value and so we have " to make a "