1. Import appropriate MathJax library in your page

Best way of importing MathJax library is using CDN links.

For example, if using TeX/LaTeX or MathML, should import following code in your page.

<script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML"></script>

If using AsciiMath, import following code.

<script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=AM_HTMLorMML-full"></script>

Tips:
You can using a custom variable in Front Matter when using Jekyll like following.

---
# layout: post
title: How to use MathJax with Jekyll
includeMathJax: true
---

And adding following codes in layout template.

{% if page.includeMathJax == true %}
   <script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=AM_HTMLorMML-full"></script>
{% endif %}

2. Use <span> or <div> for escaping markdown interpreter

Because Markdown interpreter will leave the codes alone between HTML tags like <div>, <span>. Following codes demostrate a example of math formula snippet in a post using AsciiMath syntax.

<div>
`x = (-b +- sqrt(b^2-4ac))/(2a) .`
</div>

Will output like following

`x = (-b +- sqrt(b^2-4ac))/(2a) .`

AsciiMath notation is intruduced in MathJax 2.0, and it looks tidy than others, so I choice this as default input format. Here is the syntax reference for AsciiMath.