he sees his 3 boys, as and when he can
That sentence right there tells me this man may be harboring guilt about not being able to see his own children that much and meeting yours might be adding to his guilt, especially if he's thinking about a future with you. If so, then your kids come as a package deal and maybe he can't handle that right now.
I would simply communicate to him how much it's bothering you that he doesn't seem to want to meet your kids and then make your own decision. Any guy that I date I go on and on and on about my animals just to gage his reaction. If he avoids coming over or makes jokes about how lovely my cats would look on a piece of China with an apple stuck in their mouths , then I'm quite sure I won't be seeing his carcass again any time soon.
Also, you haven't been dating that long as it is. Some women won't even let a man meet their children until after 6 months or so. It just sounds to me like he isn't that interested in having anything to do with the rest of what makes up who you are. Think about that.
Sans