Diffusers
Safetensors
English
AmusedPipeline
art
valhalla commited on
Commit
af0f5ef
·
1 Parent(s): 0effe57

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +22 -10
README.md CHANGED
@@ -282,7 +282,7 @@ accelerate launch training/training.py \
282
  --train_batch_size <batch size> \
283
  --gradient_accumulation_steps <gradient accumulation steps> \
284
  --learning_rate 1e-4 \
285
- --pretrained_model_name_or_path huggingface/amused-256 \
286
  --instance_data_dataset 'm1guelpf/nouns' \
287
  --image_key image \
288
  --prompt_key text \
@@ -323,7 +323,7 @@ accelerate launch training/training.py \
323
  --gradient_accumulation_steps <gradient accumulation steps> \
324
  --learning_rate 2e-5 \
325
  --use_8bit_adam \
326
- --pretrained_model_name_or_path huggingface/amused-256 \
327
  --instance_data_dataset 'm1guelpf/nouns' \
328
  --image_key image \
329
  --prompt_key text \
@@ -362,7 +362,7 @@ accelerate launch training/training.py \
362
  --gradient_accumulation_steps <gradient accumulation steps> \
363
  --learning_rate 8e-4 \
364
  --use_lora \
365
- --pretrained_model_name_or_path huggingface/amused-256 \
366
  --instance_data_dataset 'm1guelpf/nouns' \
367
  --image_key image \
368
  --prompt_key text \
@@ -408,7 +408,7 @@ accelerate launch training/training.py \
408
  --train_batch_size <batch size> \
409
  --gradient_accumulation_steps <gradient accumulation steps> \
410
  --learning_rate 8e-5 \
411
- --pretrained_model_name_or_path huggingface/amused-512 \
412
  --instance_data_dataset 'monadical-labs/minecraft-preview' \
413
  --prompt_prefix 'minecraft ' \
414
  --image_key image \
@@ -444,7 +444,7 @@ accelerate launch training/training.py \
444
  --train_batch_size <batch size> \
445
  --gradient_accumulation_steps <gradient accumulation steps> \
446
  --learning_rate 5e-6 \
447
- --pretrained_model_name_or_path huggingface/amused-512 \
448
  --instance_data_dataset 'monadical-labs/minecraft-preview' \
449
  --prompt_prefix 'minecraft ' \
450
  --image_key image \
@@ -480,7 +480,7 @@ accelerate launch training/training.py \
480
  --train_batch_size <batch size> \
481
  --gradient_accumulation_steps <gradient accumulation steps> \
482
  --learning_rate 1e-4 \
483
- --pretrained_model_name_or_path huggingface/amused-512 \
484
  --instance_data_dataset 'monadical-labs/minecraft-preview' \
485
  --prompt_prefix 'minecraft ' \
486
  --image_key image \
@@ -521,7 +521,7 @@ accelerate launch ./training/training.py \
521
  --mixed_precision fp16 \
522
  --report_to wandb \
523
  --use_lora \
524
- --pretrained_model_name_or_path huggingface/amused-256 \
525
  --train_batch_size 1 \
526
  --lr_scheduler constant \
527
  --learning_rate 4e-4 \
@@ -551,7 +551,7 @@ accelerate launch ./training/training.py \
551
  --mixed_precision fp16 \
552
  --report_to wandb \
553
  --use_lora \
554
- --pretrained_model_name_or_path huggingface/amused-512 \
555
  --train_batch_size 1 \
556
  --lr_scheduler constant \
557
  --learning_rate 1e-3 \
@@ -570,8 +570,20 @@ accelerate launch ./training/training.py \
570
 
571
  ## 4. Acknowledgements
572
 
573
- TODO
 
 
 
574
 
575
  ## 5. Citation
576
 
577
- TODO
 
 
 
 
 
 
 
 
 
 
282
  --train_batch_size <batch size> \
283
  --gradient_accumulation_steps <gradient accumulation steps> \
284
  --learning_rate 1e-4 \
285
+ --pretrained_model_name_or_path amused/amused-256 \
286
  --instance_data_dataset 'm1guelpf/nouns' \
287
  --image_key image \
288
  --prompt_key text \
 
323
  --gradient_accumulation_steps <gradient accumulation steps> \
324
  --learning_rate 2e-5 \
325
  --use_8bit_adam \
326
+ --pretrained_model_name_or_path amused/amused-256 \
327
  --instance_data_dataset 'm1guelpf/nouns' \
328
  --image_key image \
329
  --prompt_key text \
 
362
  --gradient_accumulation_steps <gradient accumulation steps> \
363
  --learning_rate 8e-4 \
364
  --use_lora \
365
+ --pretrained_model_name_or_path amused/amused-256 \
366
  --instance_data_dataset 'm1guelpf/nouns' \
367
  --image_key image \
368
  --prompt_key text \
 
408
  --train_batch_size <batch size> \
409
  --gradient_accumulation_steps <gradient accumulation steps> \
410
  --learning_rate 8e-5 \
411
+ --pretrained_model_name_or_path amused/amused-512 \
412
  --instance_data_dataset 'monadical-labs/minecraft-preview' \
413
  --prompt_prefix 'minecraft ' \
414
  --image_key image \
 
444
  --train_batch_size <batch size> \
445
  --gradient_accumulation_steps <gradient accumulation steps> \
446
  --learning_rate 5e-6 \
447
+ --pretrained_model_name_or_path amused/amused-512 \
448
  --instance_data_dataset 'monadical-labs/minecraft-preview' \
449
  --prompt_prefix 'minecraft ' \
450
  --image_key image \
 
480
  --train_batch_size <batch size> \
481
  --gradient_accumulation_steps <gradient accumulation steps> \
482
  --learning_rate 1e-4 \
483
+ --pretrained_model_name_or_path amused/amused-512 \
484
  --instance_data_dataset 'monadical-labs/minecraft-preview' \
485
  --prompt_prefix 'minecraft ' \
486
  --image_key image \
 
521
  --mixed_precision fp16 \
522
  --report_to wandb \
523
  --use_lora \
524
+ --pretrained_model_name_or_path amused/amused-256 \
525
  --train_batch_size 1 \
526
  --lr_scheduler constant \
527
  --learning_rate 4e-4 \
 
551
  --mixed_precision fp16 \
552
  --report_to wandb \
553
  --use_lora \
554
+ --pretrained_model_name_or_path amused/amused-512 \
555
  --train_batch_size 1 \
556
  --lr_scheduler constant \
557
  --learning_rate 1e-3 \
 
570
 
571
  ## 4. Acknowledgements
572
 
573
+ Suraj led training. William led data and supported training. Patrick supported both training and
574
+ data and provided general guidance. Robin trained the VQ-GAN and provided general guidance.
575
+ Also, immense thanks to community contributor Isamu Isozaki for helpful discussions and code
576
+ contributions.
577
 
578
  ## 5. Citation
579
 
580
+ ```
581
+ @misc{patil2024amused,
582
+ title={aMUSEd: An Open MUSE Reproduction},
583
+ author={Suraj Patil and William Berman and Robin Rombach and Patrick von Platen},
584
+ year={2024},
585
+ eprint={2401.01808},
586
+ archivePrefix={arXiv},
587
+ primaryClass={cs.CV}
588
+ }
589
+ ```