For most part life expectancy measurements did not result from the fact that people died because of old age at 25 or 35. But infant mortality rate was much higher. Think about it Aristotle died at 62 and Socrates at 70
What we are looking at is the average life expectancy, not just individual ages.
Moreso, we considered the other hominid ancestors of the extant Homo sapiens - like the Australopithecus; not just the ages of people in this era.
These average ages are; better looked upon; as something that spanned over many millennia.
Like the period of the earlier hominid specied to the Neolithic era isn't just some few years. If it is within a short period of time; then it might not necessarily be termed evolution - though some theories support catastrophism instead of gradualism. But overly; evolution is considered a slow process, which spans millions of years.
I know we are talking about average life expectancy. However, this variable is very different from ageing. Ageing effects average life expectancy of population under measurement. Let's say you are talking about average life expectancy at age 35 or 70. What it will end up telling you is a chance that most of the people in this population will live upto that age. See this data for instance. You can take into account the life expectancy at different ages. McDonald and Ruhe explains this difference very nicely, in their review publication in journal nutrition. They also talk about the evolutionary forces acting on ageing.
In fact we can ask the question that how much of average life expectancy is under the regulation of genes? In fact twin studies show that genetic contribution over here is just some 20-25%. While rest of the variation is explained by environment. I think that 20-25% should be contribution of ageing. But what is under selection? Is slow ageing selected against? Or some other benefit that also accelerates ageing tends to increase fitness? For instance take p53 selected which maintains genomic stability. However p53 achieves so by inducing either cell cycle arrest or apoptosis. Both of these can affect the ageing phenotype..
For the second part, as long as you happen to select a trait, it will fall under umbrella of evolution. And evolution can happen even as a step function. There has been huge bottleneck events in past that have shaped many species. Selection of lactose tolerance trait and fair skin in European populations did not take millions of years either. I think it took something like 8000 years.