Would it be considered fair usage if five seconds of a television show is used as part of a three-minute video?

Fair use is inherently a gray area and problematic in regards to the use of short clips. From a legal perspective, various legal and non-legal/practical issues must be considered.....just for example, the past litigiousness of the content owner, which for a TV show will usually be either a production company or the network, as well as the nature of the material used (for example, is it the use of an iconic scene from a legendary TV show?). Another consideration is how the video will be used and possibly distributed or broadcast, because in some instances a third party will require proof of licensing before showing the video. Also, you can't necessarily assume there's no loss of profit potential, as that is a more complicated legal issue than it appears to be on the surface. Sorry for the complicated answer, but that's the reality of many 'fair use' situations.

Fair use is always a question of unique facts that, when you have doubts about, ought to give pause. Ask yourself, why are you using somebody else's video?

Probably not. Not only would you be violating the owner's copyright, but also the right of publicity of any actor who appeared in the clip.