Can't get ControlNet on Inf2 to work

[FraPochetti]

Hi team,
I am adapting this notebook, essentially instantiating a ControlNet pipe, such as

controlnet = ControlNetModel.from_pretrained("DionTimmer/controlnet_qrcode-control_v1p_sd15",
                                             torch_dtype=torch.float16)

pipe = StableDiffusionControlNetImg2ImgPipeline.from_pretrained(
    "runwayml/stable-diffusion-v1-5",
    controlnet=controlnet,
    safety_checker=None,
    torch_dtype=torch.float16
)

and then going for torch_neuronx.trace
I suspect the number 1 blocker is the fact that the main libs the notebooks suggest to install

!pip install diffusers==0.14.0 transformers==4.30.2 accelerate==0.16.0 safetensors==0.3.1 matplotlib 

are too "old" for ControlNet. For instance, transformers and accelerate need to be upgraded.
This prompts the update of other dependencies and then I end up with an env that is completely different that the originally recommended one.
This causes, for instance, torch complaining about CUDA (among other things), whereas we are on Inf2 (this is confusing).
Tried multiple times but somehow couldn't get very far.
I also tried with the latest Neuron release and I can't get it to work.

Any help would be massively appreciated!

[hannanjgaws]

Hi @FraPochetti:

Thank you for raising this. Enabling ControlNet is on our roadmap and support will be enabled in a future Neuron release. We'll keep this ticket open until support is available.

[huseyintemiz]

ControlNet support (SD1.5 and SDXL) is not available yet.