Can I use copyrighted video clips for teaching in my online course?

The situation you describe falls under the "fair use" doctrine, which allows someone to use all or part of a copyrighted work without having to get the permission of the copyright owner. This doctrine is very narrow and is always dependent on exactly what you are doing and exactly what works you are using, including how much of those works you are using. "Fair use" can be permitted in the situations of commentary, search engines, criticism, news reporting, research, teaching, library archiving and scholarship, but even in these situations use of another person's work as a "fair use" is not without restrictions. One of the primary factors that works AGAINST "fair use" is whether your use is commercial in nature. Unfortunately there is a very high likelihood that your use would not be a "fair use" and you therefore would have to get permission to use the clips. Since "fair use" is so dependent on the precise facts, you should definitely consult with someone knowledgeable in this field before you make your online course available to see whether you might be inviting an allegation of copyright infringement.