If I illustrate a children’s book with cut up magazines, is it copyright infringement?

You cannot use anything that belongs to another person without permission.

If you are using or incorporating someone else's work for the benefit of your project, it is always advisable to get permission from the rights holder of the work you are using. However, if it's so unrecognizable that a reasonable person would not be able to identify the image, then you MAY have a right to claim "fair use". It is highly recommended that you consult with an attorney and have them review the material in detail and render an expert, first hand, opinion.

This would likely constitute copyright infringement. Unless you fall within one of the fair use exceptions you cannot use others work product in producing your own product.

If the images are recognizable it would be essential to obtain permission from the copyright owners, otherwise, you risk the liability of an infringement claim.

There is a copyright law term "de minimis." That means that if a very small part of a copyrighted work is used to create a derivative work without permission of the copyright owner of the original work, that it may not be copyright violation. That might apply to you using tiny parts of magazine photos. However, the pieces used may not be the ones that are the essence of the original works. So, for example, if you were to take a hole puncher and make tiny pieces of different colors out of magazine photos and use those as mosaic pieces in a collage, this might be a "de minimis" use where it does not violate the original copyrights. However, if the original works are recognizable or you are using the most essential elements and they are recognizable, then this may not be de minimis usage and may be copyright violation. In either case, there are many complications involved in your ability to own copyright on a derivative work where you are not the owner of the original parts. In your instance, it may or may not be possible. This is why you should go to a lawyer who knows this area of law and show them exactly what you are doing and get their advice. It is well worth it.

Yes, it is copyright infringement. Your use of the cut up pieces, no matter what size they are, is clearly copyright infringement. I would strongly recommend you do not do this.