took me a while to find $sin^N\left(x\right)$ expansion formula. found them here.
for odd $N$:
$\begin{eqnarray} cos^N \left( x \right) &=& \frac{1}{2^\left( N-1\right)} \left[ \sum_{k=0}^{2k < N} {N \choose 2k} cos\left( \left( N - 2k \right) x \right) \right] \\ sin^N \left( x \right) &=& \frac{\left(-1\right)^{\lfloor N/2 \rfloor}}{2^\left( N-1\right)} \left[ \sum_{k=0}^{2k < N} \left( -1 \right)^k {N \choose 2k} sin\left( \left( N - 2k \right) x \right) \right] \end{eqnarray}$
for even $N$:
$\begin{eqnarray} cos^N \left( x \right) &=& \frac{1}{2^\left( N-1\right)} \left[ \sum_{k=0}^{2k < N} {N \choose 2k} cos\left( \left( N - 2k \right) x \right) \right] + \frac{1}{2^N} {N \choose N/2} \\ sin^N \left( x \right) &=& \frac{\left(-1\right)^{N/2}}{2^\left( N-1\right)} \left[ \sum_{k=0}^{2k < N} \left( -1 \right)^k {N \choose 2k} cos\left( \left( N - 2k \right) x \right) \right] + \frac{1}{2^N} {N \choose N/2} \end{eqnarray}$
For third order nonlinearity ($N=3$):
$\begin{eqnarray} cos^3\left(x \right) &=& \frac{3}{4} cos \left( x \right) +\frac{1}{4} cos \left( 3 x \right) \end{eqnarray}$
if we have a nonlinear system $y = G x + a_3 x^3$, $G$ is linear gain and $a_3$ is the magnitude of thrid order nonlinearity, then for a single tone signal, $x = A_0 cos\left( w_0 t \right)$,
$\begin{eqnarray} y &=& \left( G + \frac{3 a_3 A_0^2}{4} \right) A_0 cos\left( w_0 t \right) + \frac{a_3 A_0^3}{4} cos \left( 3 w_0 t \right) \end{eqnarray}$
$G+ C$, where $C = \frac{3 a_3 A_0^2}{4}$ represents the gain compression (when PNA transmits a tone and measures a tone at the same frequency as the input tone; output tone power is not $G \times$ input tone power. third order nonlinearity impact the tone power at the output; it's obvious but good to clearly show that third order nonlinearity impacts fundamental power).
$\begin{eqnarray} HD_{3}^{dBc} &=& 20 \times log_{10}{\left| 3+ \frac{4 G}{a_3 A_0^2}\right|} \label{eq:hd3} \end{eqnarray}$
$\begin{eqnarray} P_{out} &=& \left( G + C\right)^2 A_0^2 \\ P_{out}^{dB} &=& dB_{20}\left( G+C \right) + P_{in}^{dB} \end{eqnarray}$
$P_{1dB}$ compression is the input or output power for which, $ \left[ dB_{20}\left( G \right) + P_{in}^{dB} \right] - P_{out}^{dB} = 1$.
$\begin{eqnarray} dB_{20}\left( \frac{G}{G+C} \right) &=& 1 \\ \frac{G}{G + C} &=& 1.122 \\ 1+\frac{C}{G} &=& 0.8913 \\ C &=& -0.1087 \times G\\ A_0^2 &=& \frac{0.145 \times G}{\left|a_3\right|} \label{eq:amp} \\ P_{in}^{1dB} &=& -1.4dBm + dB_{10}\left( G\right) - dB_{10}\left( \left|a_3\right| \right) \end{eqnarray}$
Substituting Eq.$~\ref{eq:amp}$ in Eq.$~\ref{eq:hd3}$ results in HD3 of $-27.8~dBc$ at $P_{in}^{1dB}$ input power level.
