File size: 2,463 Bytes
7dc84d4 a74c434 7dc84d4 92814b1 7dc84d4 11f2343 7dc84d4 5888da8 11f2343 43fddf2 11f2343 7dc84d4 11f2343 7dc84d4 11f2343 62d2d12 11f2343 7dc84d4 aec79eb bf3bb91 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 |
---
license: creativeml-openrail-m
library_name: diffusers
tags:
- stable-diffusion
- stable-diffusion-diffusers
- text-to-image
- diffusers
- diffusers-training
- lora
- stable-diffusion
- stable-diffusion-diffusers
- text-to-image
- diffusers
- diffusers-training
- lora
base_model: stabilityai/stable-diffusion-2-1
inference: true
---
# LoRA text2image fine-tuning - remi349/sd_trained_3D_lora
These are LoRA adaption weights are for stabilityai/stable-diffusion-2-1. The weights were fine-tuned on the remi349/finetuning_dataset_for_3D_training dataset thanks to the library [diffusers](https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/train_text_to_image_lora.py).
## Intended uses & limitations
This model aims at generating images of isolated objects, compatible with 2D_to_3D models like [Triposr](https://github.com/VAST-AI-Research/TripoSR) or [CRM](https://huggingface.co/Zhengyi/CRM).
It was finetuned in order to create after a pipeline of prompt-to-3D model.
#### How to use
```python
# First load the basic architecture and everything
import torch
from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2-1", torch_dtype=torch.float16)
# Then add the lora weights to the model stable diffusion 2
pipe.unet.load_attn_procs('ACROSS-Lab/PromptTo3D_sd_finetuned')
pipe.to("cuda")
# Then you can begin the inference process on a prompt and save the image generated
prompt = 'a rabbit with a yellow jacket'
image = pipe(prompt, num_inference_steps=30, guidance_scale=7.5).images[0]
image.save("my_image.png")
```
#### Limitations and bias
This model is a first try some hyperparameters tuning should be done, but for that we would need a solid automated benchmark.
## Training details
The model finetuned model is [Stable Diffusion 2](https://huggingface.co/stabilityai/stable-diffusion-2).
The data used to train this model is the dataset available on uggingface at 'remi349/finetuning_dataset_for_3D_training'.
you can download it thanks to the command
```python
from datasets import load_dataset
dataset = load_dataset("ACROSS-Lab/PromptTo3D_sd_dataset", split = 'train')
```
This dataset is a subset of the dataset [Objaverse](https://objaverse.allenai.org/).
## Collaboration
This model and dataset has been made in collaboration by [Josué ADOSSEHOUN](https://huggingface.co/josh007) and [Rémi DUCOTTET](https://huggingface.co/remi349) |