Avoda Zara 53b - When Did E.Y. Become Ours?

The very first and one of Rashi's most famous commentaries on the Torah is that the Torah began with parshas breishis to establish the claim of the Jews to the land of Israel - כל הארץ של הקב"ה היא, הוא בראה ונתנה לאשר ישר בעיניו, ברצונו נתנה להם, וברצונו נטלה מהם ונתנה לנו

But, it is not clear at what point in history are we considered to have taken ownership of Eretz Yisroel. See this post where I once explained the argument between the shepherds of Avrohom and Lot. They both could have held that ownership was at the point where Avrohom made a "chazaka" on EY, even before we actually conquered it. Our gemara strongly supports this view. The gemara says that the asheira trees in E.Y. had status of Avoda Zara, but the gemara doesn't understand why since the Jews "owned" E.Y. from the time of Avrohom - ירושה היא להם מאבותיהם, so the goyim shouldn't have had the ability to make it assur since אין אדם אוסר דבר שאינו שלו? The gemara responds that at the point when the Jews sinned by the golden calf they indicated that they were interested in the worship of the idolatry in E.Y. Clearly, the gemara holds that E.Y. belonged to klal yisroel from the time of Avrohom, even before they actually conquered it. This is also the approach of the Ramban in parshas lech lecha (bris bein ha'besarim) - אמר הכתוב בפעם הראשונה לזרעך אתן בלשון עתיד וכן בשנית כי עד הנה לא נתנה אליו כולה ולפיכך אמר לו אתננה אבל בשלישית בשעת הברית אמר לזרעך נתתי לאמר שיכרות לו ברית על המתנה שכבר נתן לו וכן בעת המילה כשאמר "לאחזת עולם" אמר לו "ונתתי לך" בעתיד ורש"י כתב לזרעך נתתי אמירת הגבוה כמו שהיא עשויה ואין צורך במקום הזה - The approach of the Ramban is that Hashem had already actually given E.Y. to Avrohom's descendants, unlike Rashi who says that Hashem's promise is as good as done.