garden plots dont sisapear after you remove everything from them and stop interacting with them i abandoned a plot in order to place a watchtower in its position 3 days later its still there.