David Letterman on Monday kicked off one of his final weeks as host of CBS's Late Show with quite the lineup: President Obama stopped by for one last chat, while Will Ferrell delivered his Harry Caray impersonation.

The show closed with Brandi Carlile and The Avett Brothers taking the stage to perform one of Letterman's favorite songs and "Keep on the Sunny Side."

Video of that performance can be seen below.

Letterman's final episode is set for May 20.