Optimize SD3.5 Large Controlnets

Not only the size of the files is large, but using ControlNets makes running a workflow significantly slower with SD3.5L. I think especially with ControlNets SD3.5L can have a good place in our workflows, but the speed has been a big discouragement for me to experiment enough.