More and more ancestors, at each generation, will drop out of the picture as you move to more distant ancestors. This applies only to the chromosome under consideration. These ancestors may well be found on other chromosomes. Because the DNA is random, many of your ancestors will be represented on some chromosomes, for many more generations.