(a) If f(x) is to be a proper density function, then its integral over the given support must evaulate to 1:
[tex]\displaystyle\int_{-\infty}^\infty f(x)\,\mathrm dx = \int_0^\infty cxe^{-x^2}\,\mathrm dx=1[/tex]
For the integral, substitute u = x ² and du = 2x dx. Then as x → 0, u → 0; as x → ∞, u → ∞:
[tex]\displaystyle\frac12\int_0^\infty ce^{-u}\,\mathrm du=\frac c2\left(\lim_{u\to\infty}(-e^{-u})-(-1)\right)=1[/tex]
which reduces to
c / 2 (0 + 1) = 1 → c = 2
(b) Find the probability P(1 < X < 3) by integrating the density function over [1, 3] (I'll omit the steps because it's the same process as in (a)):
[tex]\displaystyle\int_1^3 2xe^{-x^2}\,\mathrm dx = \boxed{\frac{e^8-1}{e^9}} \approx 0.3678[/tex]
