Early astronomers were able to tell the difference between planets and stars because planets in our Solar System appear to move in complicated paths across the sky, but stars don't.
That is, if you observe the sky night after night, the stars will all appear in fixed positions with respect to each other. They will rise and set a few minutes earlier each night (an effect that is due to the Earth's motion around the Sun), but otherwise nothing will change. This is why the background stars are sometimes referred to as the "celestial sphere" -- from our point of view, it looks like the stars are "painted" onto a gigantic sphere that surrounds Earth and therefore are unable to move with respect to each other.
Planets, on the other hand, are observed to move in very complicated paths with respect to the background stars, sometimes even appearing to go "against the grain" and reverse their directions. Therefore, they are easily distinguishable from stars if you look at the sky night after night. Although ancient astronomers did not have a correct explanation for this phenomenon, we now know that the complicated motion is just a projection effect -- it is due to the fact that Earth and the other planets are physically moving in orbits around the Sun, so the planets' relative positions as seen from Earth (with respect to the fixed background stars) change as time goes on.
There are other observational differences between planets and stars too, by the way -- such as the fact that planets almost never twinkle. (See the related question on this.)