Only the fusion of hydrogen into helium can provide enough energy to power most stars. In our models we find that choosing how much helium (and the trace amount of heavier elements) almost completely determines a star's temperature and luminosity. The models that fit Red Giant's characteristics suggest that they are mostly helium inside and so they must be old; the hydrogen in their cores has already been turned into helium. The models that fit White Dwarfs low luminosity and temperatures best suggest that no energy generation is taking place and that their cores are made of elements heavier than helium.
So, in the end, we can make an educated guess about the age of a star by the core composition of the model that fits best--and the time it would take the hydrogen to turn into the helium. This takes billions of years for main sequences stars like the Sun. To turn helium into carbon and heavier elements takes much less time (only a few hundred million to a billion years), and it is these models that fit the observed characteristics of Red Giant stars best. So we conclude that Red Giants are old stars, which don't have much lifetime left.
For hydrogen-burning stars on the main sequence, such as the Sun, there is another way to narrow down an age. Just as the Sun has an 11 year sunspot cycle, stars also have activity cycles; they have many "star-spots" sometimes, and a few at other times. These cycles are detectable by looking for the spectral features that active surface regions emit, such as the emission lines of the common element calcium. Models suggest that the activity of a star, and the brightness of these spectral features, declines as a star ages. Hence, one way of determining a main sequence star's age is to measure how bright these activity-sensitive spectral features are.
How about the age of a White Dwarf star? These stars aren't making their own energy anymore, and only shine because they are still hot from their hydrogen and helium burning phases. They are so small and so hot that it takes billions of years for them to cool to the temperature of interstellar space, which is just a few degrees above absolute zero.
Think of a cup of coffee. When first poured, it is very hot, but as time goes on the temperature falls. If you know how fast a cup of coffee cools, you can measure its present temperature and determine how long it has been since it was poured. The color of a White Dwarf is easy to measure, and it directly tells us its temperature. The redder it is, the cooler it is--therefore the older it is.
Curiously, we find no White Dwarfs cooler than about 4,000 Kelvins. It takes a White Dwarf about 10 billion years to cool to this temperature. So we conclude that even the first generation of stars in our galaxy, whose remnants are now White Dwarfs, have not had a chance to cool below 4,000 Kelvins. By that reckoning, the galaxy, and hence the whole universe, must be at least 10 billion years old.