I simply added
to my Header.php.
Update: I made a plugin to conditionally insert the code above into pages that contain grave accents.
Here is an example. I wrote
\` e^(i pi) = -1 \` and it produces ` e^(i pi) = -1 `.
I modified Peter's script to disable the $ as a delimiter since I tend to use those in other contexts.