There we already some talks about Captain America having his sequel with both Stephen McFeely and Christopher Markus, the writers of The First Avenger, hired to write the script.
Same goes for Thor that without knowing whether it will be a success or not, Marvel Studios President Kevin Feige seems quite confident.
In an interview with D23 (via IGN) Feige confirmed that sequels for both Thor and Captain America: The First Avenger are in the works. Little was said of Thor but he did confirm that the next Cap movie will be a movie set in the present.
They are still trying to work out solo movies for characters that have only appeared in other projects thus far, including Black Widow, Hawkeye, and S.H.I.E.L.D.
If Thor is a massive hit, and there are good reasons to believe so, for instance we have Natalie Portman and the movie seemed to be made for the masses. If it works well in the box office, thinking of a sequel might be a good idea, but if it’s a disaster it could present a bit of a problem.