It depends on the initial entangled state.
If you create a state
(|up>|down> + |down>|up>)/sqrt(2),
And you measure the first particle to be up, then the second particle is automatically in the state |down>, i.e. not in a superposition state in this basis.
Both particles are in a superposition of both states until you measure one, upon which its superposition collapses randomly to one value, and the other instantly collapses to the opposite one. So your second answer is right. There are a lot of fun theories as to why, but all we can say for sure is that it is true.