First we can say that since there is no external force on this system so momentum is always conserved.
[tex]m_1v_{1i} + m_2v_{2i} = m_1v_{1f} + m_2v_{2f}[/tex]
[tex]0.6*0.8 + 1.2*0 = 0.6*v_{1f} + 1.2*v_{2f}[/tex]
[tex]0.48= 0.6*v_{1f} + 1.2*v_{2f}[/tex]
[tex]0.8 = v_{1f} + 2v_{2f}[/tex]
now by the condition of elastic collision
[tex]v_{2f} - v_{1f} = 0.8 - 0[\tex]
now add two equations
[tex]3*v_{2f} = 1.6[/tex]
[tex]v_{2f} = 0.533 m/s[/tex]
also from above equation we have
[tex]v_{1f} = -0.267 m/s[/tex]
So ball of mass 0.6 kg will rebound back with speed 0.267 m/s and ball of mass 1.2 kg will go forwards with speed 0.533 m/s.