There are actually two different approaches to rotating objects. One method uses quaternions, but the math is complex. The other method is very similar to that of scaling and translating, because it uses a similar matrix equation. This second method is called Euler rotation (pronounced “oiler rotation,” not “yewler”). You'll set up a matrix equation first in 2D and then in 3D. As soon as the matrix equation is set up, all you have to do is plug in the vertices one at a time and multiply the matrices to find the new position.
where θ is the angle of rotation.
As soon as you know the angle of rotation, you can take the sine ...