“The Late Show with Stephen Colbert” just has a packed slate of guests for his live episode Sunday following the Super Bowl.

Colbert is the first late night host to directly follow the game; that enviable time slot is usually reserved for promising young shows the network wants to give a big ratings boost. The Super Bowl pulled in a record 114 million viewers last year, and it’s about the best lead-in you could ask for, but it’s not a ratings guarantee.

Let’s look at how Super Bowl ratings have grown over the years, along with the available ratings for programs that aired afterword, from TV by the Numbers:

The TV landscape and viewing habits have changed over time, so lets zoom into the past 25 years and focus on the really important number: what portion of the Super Bowl audience was the network able to keep tuned in to their next show?

That number has varied quite a bit, as ratings for the game itself have broken new records almost every year. So what makes for a good lead-out show? Here’s a brief history: