deepcrayon
/
tinyrocs
1
0
Fork 0
Code Issues Activity

most tinygrad examples

main
Jeff Moe 2024-02-06 12:29:23 -07:00
parent abd4cb6012
commit 4f71126cfb
12 changed files with 654 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/_source/_static/_output/tinygrad/train_efficientnet.py.txt
View File

File diff suppressed because one or more lines are too long

20
docs/_source/_static/_output/tinygrad/train_resnet.py.txt 100644
View File

File diff suppressed because one or more lines are too long

44
docs/_source/_static/_output/tinygrad/transformer.py.txt 100644
View File

@ -0,0 +1,44 @@
0%| | 0/50 [00:00<?, ?it/s] loss 2.35 accuracy 0.05: 0%| | 0/50 [00:07<?, ?it/s] loss 2.35 accuracy 0.05: 2%|▏ | 1/50 [00:07<06:05, 7.47s/it] loss 2.26 accuracy 0.17: 2%|▏ | 1/50 [00:09<06:05, 7.47s/it] loss 2.26 accuracy 0.17: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 2.19 accuracy 0.18: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 2.21 accuracy 0.20: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 2.15 accuracy 0.22: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 2.09 accuracy 0.27: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 2.02 accuracy 0.31: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 1.99 accuracy 0.30: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 1.93 accuracy 0.30: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 1.92 accuracy 0.33: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 1.83 accuracy 0.37: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 1.77 accuracy 0.39: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 1.69 accuracy 0.43: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 1.52 accuracy 0.58: 4%|▍ | 2/50 [00:09<03:24, 4.26s/it] loss 1.52 accuracy 0.58: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 1.44 accuracy 0.65: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 1.31 accuracy 0.68: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 1.21 accuracy 0.73: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 1.08 accuracy 0.77: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 1.01 accuracy 0.79: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.93 accuracy 0.85: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.91 accuracy 0.85: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.76 accuracy 0.86: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.67 accuracy 0.85: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.72 accuracy 0.85: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.66 accuracy 0.86: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.62 accuracy 0.86: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.55 accuracy 0.87: 28%|██▊ | 14/50 [00:09<00:14, 2.55it/s] loss 0.55 accuracy 0.87: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.66 accuracy 0.85: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.57 accuracy 0.86: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.53 accuracy 0.87: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.67 accuracy 0.85: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.55 accuracy 0.85: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.51 accuracy 0.86: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.47 accuracy 0.87: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.53 accuracy 0.85: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.52 accuracy 0.85: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.49 accuracy 0.86: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.48 accuracy 0.87: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.49 accuracy 0.86: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.47 accuracy 0.86: 54%|█████▍ | 27/50 [00:09<00:03, 5.97it/s] loss 0.47 accuracy 0.86: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.48 accuracy 0.86: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.45 accuracy 0.87: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.44 accuracy 0.86: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.47 accuracy 0.86: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.49 accuracy 0.85: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.41 accuracy 0.88: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.43 accuracy 0.87: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.42 accuracy 0.87: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.40 accuracy 0.87: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.38 accuracy 0.86: 80%|████████ | 40/50 [00:09<00:00, 10.55it/s] loss 0.38 accuracy 0.86: 100%|██████████| 50/50 [00:09<00:00, 5.07it/s]
0%| | 0/16 [00:00<?, ?it/s] 6%|▋ | 1/16 [00:01<00:26, 1.77s/it] 62%|██████▎ | 10/16 [00:01<00:00, 7.23it/s] 100%|██████████| 16/16 [00:03<00:00, 4.55it/s] 100%|██████████| 16/16 [00:03<00:00, 4.27it/s]
test set accuracy is 0.868250
reducing lr to 0.0025
0%| | 0/50 [00:00<?, ?it/s] loss 0.39 accuracy 0.87: 0%| | 0/50 [00:00<?, ?it/s] loss 0.38 accuracy 0.87: 0%| | 0/50 [00:00<?, ?it/s] loss 0.38 accuracy 0.87: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.53 accuracy 0.83: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.45 accuracy 0.85: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.44 accuracy 0.85: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.43 accuracy 0.85: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.40 accuracy 0.88: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.38 accuracy 0.86: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.41 accuracy 0.86: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.43 accuracy 0.85: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.39 accuracy 0.87: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.41 accuracy 0.85: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.40 accuracy 0.86: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.42 accuracy 0.86: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.39 accuracy 0.86: 4%|▍ | 2/50 [00:00<00:05, 9.08it/s] loss 0.39 accuracy 0.86: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.39 accuracy 0.86: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.37 accuracy 0.87: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.40 accuracy 0.86: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.34 accuracy 0.89: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.37 accuracy 0.87: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.39 accuracy 0.86: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.38 accuracy 0.88: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.35 accuracy 0.88: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.37 accuracy 0.87: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.37 accuracy 0.86: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.36 accuracy 0.87: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.36 accuracy 0.88: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.36 accuracy 0.87: 30%|███ | 15/50 [00:00<00:00, 56.16it/s] loss 0.36 accuracy 0.87: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.39 accuracy 0.86: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.37 accuracy 0.87: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.35 accuracy 0.87: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.37 accuracy 0.86: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.35 accuracy 0.87: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.37 accuracy 0.86: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.35 accuracy 0.88: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.34 accuracy 0.88: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.35 accuracy 0.88: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.35 accuracy 0.88: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.34 accuracy 0.89: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.32 accuracy 0.88: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.31 accuracy 0.89: 56%|█████▌ | 28/50 [00:00<00:00, 82.18it/s] loss 0.31 accuracy 0.89: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.34 accuracy 0.89: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.33 accuracy 0.89: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.32 accuracy 0.89: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.33 accuracy 0.88: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.35 accuracy 0.89: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.32 accuracy 0.90: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.34 accuracy 0.89: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.31 accuracy 0.90: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.33 accuracy 0.89: 82%|████████▏ | 41/50 [00:00<00:00, 98.03it/s] loss 0.33 accuracy 0.89: 100%|██████████| 50/50 [00:00<00:00, 84.11it/s]
0%| | 0/16 [00:00<?, ?it/s] 62%|██████▎ | 10/16 [00:00<00:00, 91.08it/s] 100%|██████████| 16/16 [00:00<00:00, 91.47it/s]
test set accuracy is 0.893917
reducing lr to 0.0021
0%| | 0/50 [00:00<?, ?it/s] loss 0.32 accuracy 0.89: 0%| | 0/50 [00:00<?, ?it/s] loss 0.38 accuracy 0.87: 0%| | 0/50 [00:00<?, ?it/s] loss 0.38 accuracy 0.87: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.39 accuracy 0.86: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.34 accuracy 0.89: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.31 accuracy 0.89: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.33 accuracy 0.89: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.31 accuracy 0.89: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.34 accuracy 0.89: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.34 accuracy 0.88: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.35 accuracy 0.88: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.31 accuracy 0.90: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.32 accuracy 0.89: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.30 accuracy 0.90: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.30 accuracy 0.90: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.29 accuracy 0.89: 4%|▍ | 2/50 [00:00<00:04, 11.41it/s] loss 0.29 accuracy 0.89: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.28 accuracy 0.89: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.29 accuracy 0.89: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.27 accuracy 0.90: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.28 accuracy 0.88: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.32 accuracy 0.89: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.26 accuracy 0.90: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.29 accuracy 0.90: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.26 accuracy 0.92: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.29 accuracy 0.90: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.26 accuracy 0.91: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.26 accuracy 0.91: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.26 accuracy 0.92: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.23 accuracy 0.92: 30%|███ | 15/50 [00:00<00:00, 64.02it/s] loss 0.23 accuracy 0.92: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.24 accuracy 0.91: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.23 accuracy 0.93: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.24 accuracy 0.92: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.21 accuracy 0.92: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.21 accuracy 0.92: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.22 accuracy 0.92: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.19 accuracy 0.92: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.21 accuracy 0.94: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.22 accuracy 0.93: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.22 accuracy 0.92: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.21 accuracy 0.93: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.20 accuracy 0.93: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.23 accuracy 0.92: 56%|█████▌ | 28/50 [00:00<00:00, 89.01it/s] loss 0.23 accuracy 0.92: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.18 accuracy 0.95: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.20 accuracy 0.93: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.18 accuracy 0.93: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.18 accuracy 0.93: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.13 accuracy 0.96: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.18 accuracy 0.94: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.21 accuracy 0.95: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.26 accuracy 0.93: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.20 accuracy 0.93: 82%|████████▏ | 41/50 [00:00<00:00, 102.92it/s] loss 0.20 accuracy 0.93: 100%|██████████| 50/50 [00:00<00:00, 90.64it/s]
0%| | 0/16 [00:00<?, ?it/s] 38%|███▊ | 6/16 [00:00<00:00, 59.78it/s] 100%|██████████| 16/16 [00:00<00:00, 79.86it/s] 100%|██████████| 16/16 [00:00<00:00, 76.92it/s]
test set accuracy is 0.941333
reducing lr to 0.0017
0%| | 0/50 [00:00<?, ?it/s] loss 0.21 accuracy 0.92: 0%| | 0/50 [00:00<?, ?it/s] loss 0.19 accuracy 0.93: 0%| | 0/50 [00:00<?, ?it/s] loss 0.19 accuracy 0.93: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.15 accuracy 0.95: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.13 accuracy 0.96: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.18 accuracy 0.94: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.21 accuracy 0.93: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.18 accuracy 0.94: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.16 accuracy 0.95: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.16 accuracy 0.94: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.15 accuracy 0.96: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.14 accuracy 0.96: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.10 accuracy 0.97: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.14 accuracy 0.96: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.17 accuracy 0.94: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.11 accuracy 0.96: 4%|▍ | 2/50 [00:00<00:04, 11.49it/s] loss 0.11 accuracy 0.96: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.13 accuracy 0.96: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.17 accuracy 0.95: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.14 accuracy 0.95: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.10 accuracy 0.97: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.10 accuracy 0.96: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.13 accuracy 0.96: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.08 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.12 accuracy 0.95: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.11 accuracy 0.97: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.12 accuracy 0.95: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.10 accuracy 0.97: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.14 accuracy 0.94: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.08 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.08 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.12 accuracy 0.96: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.07 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.08 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.08 accuracy 0.97: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.08 accuracy 0.97: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.08 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.08 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.08 accuracy 0.97: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.08 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.07 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.07 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.09 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.05 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.16it/s] loss 0.05 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.07 accuracy 0.97: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.08 accuracy 0.98: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.10 accuracy 0.96: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.06 accuracy 0.98: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.06 accuracy 0.98: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.10 accuracy 0.97: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.06 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.08 accuracy 0.97: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.06 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.00it/s] loss 0.06 accuracy 0.99: 100%|██████████| 50/50 [00:00<00:00, 90.76it/s]
0%| | 0/16 [00:00<?, ?it/s] 56%|█████▋ | 9/16 [00:00<00:00, 89.80it/s] 100%|██████████| 16/16 [00:00<00:00, 76.47it/s]
test set accuracy is 0.981167
reducing lr to 0.0014
0%| | 0/50 [00:00<?, ?it/s] loss 0.06 accuracy 0.98: 0%| | 0/50 [00:00<?, ?it/s] loss 0.09 accuracy 0.97: 0%| | 0/50 [00:00<?, ?it/s] loss 0.09 accuracy 0.97: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.08 accuracy 0.97: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.09 accuracy 0.97: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.07 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.06 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.07 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.05 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.07 accuracy 0.97: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.06 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.10 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.06 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.08 accuracy 0.97: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.06 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.06 accuracy 0.98: 4%|▍ | 2/50 [00:00<00:04, 11.46it/s] loss 0.06 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.08 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.12 accuracy 0.96: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.06 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.08 accuracy 0.97: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.06 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.06 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.07 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.06 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.05 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.05 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.05 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.04 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.05 accuracy 0.98: 30%|███ | 15/50 [00:00<00:00, 64.21it/s] loss 0.05 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.04 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.04 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.04 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.06 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.02 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.06 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.10 accuracy 0.98: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.03 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.03 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.05 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.06 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.03 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.03 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 89.20it/s] loss 0.03 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.03 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.05 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.03 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.03 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.03 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.02 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.04 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.06 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.03 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 103.32it/s] loss 0.03 accuracy 0.99: 100%|██████████| 50/50 [00:00<00:00, 91.01it/s]
0%| | 0/16 [00:00<?, ?it/s] 62%|██████▎ | 10/16 [00:00<00:00, 91.17it/s] 100%|██████████| 16/16 [00:00<00:00, 91.00it/s]
test set accuracy is 0.993833
reducing lr to 0.0012
0%| | 0/50 [00:00<?, ?it/s] loss 0.04 accuracy 0.99: 0%| | 0/50 [00:00<?, ?it/s] loss 0.04 accuracy 0.99: 2%|▏ | 1/50 [00:00<00:05, 8.52it/s] loss 0.05 accuracy 0.99: 2%|▏ | 1/50 [00:00<00:05, 8.52it/s] loss 0.08 accuracy 0.97: 2%|▏ | 1/50 [00:00<00:05, 8.52it/s] loss 0.05 accuracy 0.99: 2%|▏ | 1/50 [00:00<00:05, 8.52it/s] loss 0.05 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.05 accuracy 0.98: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.04 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.08 accuracy 0.98: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.04 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.04 accuracy 0.98: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.04 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.02 accuracy 1.00: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.05 accuracy 0.98: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.03 accuracy 1.00: 8%|▊ | 4/50 [00:00<00:02, 19.79it/s] loss 0.03 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.03 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.02 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.04 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.02 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.05 accuracy 0.98: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.04 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.02 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.02 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.03 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.02 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.02 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.03 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.94it/s] loss 0.03 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.02 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.02 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.03 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.03 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.02 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.02 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.02 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.03 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.02 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 90.27it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.78it/s] loss 0.02 accuracy 0.99: 86%|████████▌ | 43/50 [00:00<00:00, 103.78it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.78it/s] loss 0.02 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.78it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.78it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.78it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.78it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.78it/s] loss 0.01 accuracy 1.00: 100%|██████████| 50/50 [00:00<00:00, 85.93it/s]
0%| | 0/16 [00:00<?, ?it/s] 62%|██████▎ | 10/16 [00:00<00:00, 90.17it/s] 100%|██████████| 16/16 [00:00<00:00, 90.08it/s]
test set accuracy is 0.996000
reducing lr to 0.0010
0%| | 0/50 [00:00<?, ?it/s] loss 0.03 accuracy 0.99: 0%| | 0/50 [00:00<?, ?it/s] loss 0.03 accuracy 0.99: 2%|▏ | 1/50 [00:00<00:05, 8.20it/s] loss 0.05 accuracy 0.98: 2%|▏ | 1/50 [00:00<00:05, 8.20it/s] loss 0.06 accuracy 0.99: 2%|▏ | 1/50 [00:00<00:05, 8.20it/s] loss 0.04 accuracy 1.00: 2%|▏ | 1/50 [00:00<00:05, 8.20it/s] loss 0.04 accuracy 1.00: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.02 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.08 accuracy 0.97: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.02 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.04 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.02 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.02 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.02 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.03 accuracy 0.99: 8%|▊ | 4/50 [00:00<00:02, 19.49it/s] loss 0.03 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.02 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.02 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.02 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.02 accuracy 0.99: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 34%|███▍ | 17/50 [00:00<00:00, 66.35it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.02 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.03 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 1.00: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 0.99: 60%|██████ | 30/50 [00:00<00:00, 89.87it/s] loss 0.01 accuracy 0.99: 86%|████████▌ | 43/50 [00:00<00:00, 103.28it/s] loss 0.00 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.28it/s] loss 0.02 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.28it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.28it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.28it/s] loss 0.01 accuracy 1.00: 86%|████████▌ | 43/50 [00:00<00:00, 103.28it/s] loss 0.01 accuracy 0.99: 86%|████████▌ | 43/50 [00:00<00:00, 103.28it/s] loss 0.02 accuracy 0.99: 86%|████████▌ | 43/50 [00:00<00:00, 103.28it/s] loss 0.02 accuracy 0.99: 100%|██████████| 50/50 [00:00<00:00, 85.26it/s]
0%| | 0/16 [00:00<?, ?it/s] 56%|█████▋ | 9/16 [00:00<00:00, 89.31it/s] 100%|██████████| 16/16 [00:00<00:00, 89.70it/s]
test set accuracy is 0.999917
reducing lr to 0.0008
0%| | 0/50 [00:00<?, ?it/s] loss 0.01 accuracy 0.99: 0%| | 0/50 [00:00<?, ?it/s] loss 0.03 accuracy 0.99: 0%| | 0/50 [00:00<?, ?it/s] loss 0.03 accuracy 0.99: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 0.99: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.03 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.02 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.59it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.04 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.02 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.03 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 57.90it/s] loss 0.03 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.02 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.04 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.50it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.03 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.02 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.68it/s] loss 0.01 accuracy 1.00: 100%|██████████| 50/50 [00:00<00:00, 85.24it/s]
0%| | 0/16 [00:00<?, ?it/s] 62%|██████▎ | 10/16 [00:00<00:00, 90.34it/s] 100%|██████████| 16/16 [00:00<00:00, 91.04it/s]
test set accuracy is 0.999583
reducing lr to 0.0007
0%| | 0/50 [00:00<?, ?it/s] loss 0.00 accuracy 1.00: 0%| | 0/50 [00:00<?, ?it/s] loss 0.00 accuracy 1.00: 0%| | 0/50 [00:00<?, ?it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.02 accuracy 0.99: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.02 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:05, 9.42it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.02 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.02 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.03 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 57.33it/s] loss 0.03 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.01 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.01 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.02 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 83.23it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.02 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.01 accuracy 0.99: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 98.74it/s] loss 0.00 accuracy 1.00: 100%|██████████| 50/50 [00:00<00:00, 85.08it/s]
0%| | 0/16 [00:00<?, ?it/s] 62%|██████▎ | 10/16 [00:00<00:00, 91.26it/s] 100%|██████████| 16/16 [00:00<00:00, 91.49it/s]
test set accuracy is 0.999833
reducing lr to 0.0006
0%| | 0/50 [00:00<?, ?it/s] loss 0.00 accuracy 1.00: 0%| | 0/50 [00:00<?, ?it/s] loss 0.01 accuracy 0.99: 0%| | 0/50 [00:00<?, ?it/s] loss 0.01 accuracy 0.99: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.02 accuracy 0.99: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.01 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.01 accuracy 0.99: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 4%|▍ | 2/50 [00:00<00:04, 11.42it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.01 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.01 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.02 accuracy 0.99: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.01 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 30%|███ | 15/50 [00:00<00:00, 63.95it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.03 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.02 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.02 accuracy 0.99: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.02 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.01 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.02 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.00 accuracy 1.00: 56%|█████▌ | 28/50 [00:00<00:00, 88.72it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.01 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.00 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.02 accuracy 1.00: 82%|████████▏ | 41/50 [00:00<00:00, 102.54it/s] loss 0.02 accuracy 1.00: 100%|██████████| 50/50 [00:00<00:00, 90.37it/s]
0%| | 0/16 [00:00<?, ?it/s] 44%|████▍ | 7/16 [00:00<00:00, 63.65it/s] 100%|██████████| 16/16 [00:00<00:00, 77.29it/s]
test set accuracy is 0.999750
reducing lr to 0.0005
04 + 04 = 004 (correct: 008)
09 + 09 = 019 (correct: 018)
00 + 10 = 000 (correct: 010)
Wrong predictions: 3, acc = 0.9998

30
docs/_source/_static/_output/tinygrad/vgg7.py.txt 100644
View File

@ -0,0 +1,30 @@
python3 -m examples.vgg7 import MODELJSON MODEL
imports a waifu2x JSON vgg_7 model, i.e. waifu2x/models/vgg_7/art/scale2.0x_model.json
into a safetensors file
weight tensors are ordered in tinygrad/ncnn form, as so: (outC,inC,H,W)
*this format is used by most other commands in this program*
python3 -m examples.vgg7 import_kinne MODEL_KINNE MODEL_SAFETENSORS
imports a model in 'KINNE' format (raw floats: used by older versions of this example) into safetensors
python3 -m examples.vgg7 execute MODEL IMG_IN IMG_OUT
given an already-nearest-neighbour-scaled image, runs vgg7 on it
output image has 7 pixels removed on all edges
do not run on large images, will have *hilarious* RAM use
python3 -m examples.vgg7 execute_full MODEL IMG_IN IMG_OUT
does the 'whole thing' (padding, tiling)
safe for large images, etc.
python3 -m examples.vgg7 new MODEL
creates a new model (experimental)
python3 -m examples.vgg7 train MODEL SAMPLES_DIR ROUNDS ROUNDS_SAVE
trains a model (experimental)
(how experimental? well, every time I tried it, it flooded w/ NaNs)
note: ROUNDS < 0 means 'forever'. ROUNDS_SAVE <= 0 is not a good idea.
expects roughly execute's input as SAMPLES_DIR/IDXa.png
expects roughly execute's output as SAMPLES_DIR/IDXb.png
(i.e. my_samples/0a.png is the first pre-nearest-scaled image,
my_samples/0b.png is the first original image)
in addition, SAMPLES_DIR/samples_count.txt indicates sample count
won't pad or tile, so keep image sizes sane
python3 -m examples.vgg7 samplify IMG_A IMG_B SAMPLES_DIR SIZE
creates overlapping micropatches (SIZExSIZE w/ 7-pixel border) for training
maintains/creates samples_count.txt automatically
unlike training, IMG_A must be exactly half the size of IMG_B

1
docs/_source/_static/_output/tinygrad/vit.py.txt 100644
View File

@ -0,0 +1 @@
(1, 1000) 208 16.274181 Labrador retriever

16
docs/_source/_static/_output/tinygrad/vits.py.txt 100644
View File

@ -0,0 +1,16 @@
INFO:root:Model has 109 speakers
INFO:root:You selected speaker 6 (name: ?)
Traceback (most recent call last):
File "/home/jebba/devel/tinygrad/tinygrad/examples/vits.py", line 723, in <module>
net_g = load_model(text_mapper.symbols, hps, model_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/examples/vits.py", line 535, in load_model
_ = load_checkpoint(fetch(model[1]), net_g, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/examples/vits.py", line 540, in load_checkpoint
checkpoint_dict = torch_load(checkpoint_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/nn/state.py", line 145, in torch_load
_, _, _, rwd, _, ids, base_offset = pkl.load(), pkl.load(), pkl.load(), f.tell(), pkl.load(), pkl.load(), f.tell()
^^^^^^^^^^
_pickle.UnpicklingError: invalid load key, '<'.

100
docs/_source/_static/_output/tinygrad/whisper.py.txt 100644
View File

File diff suppressed because one or more lines are too long

10
docs/_source/_static/_output/tinygrad/yolov3.py.txt 100644
View File

File diff suppressed because one or more lines are too long

340
docs/_source/_static/_output/tinygrad/yolov8-onnx.py.txt 100644
View File

@ -0,0 +1,340 @@
Downloading https://github.com/ultralytics/assets/releases/download/v8.1.0/yolov8n-seg.pt to 'yolov8n-seg.pt'...
0%| | 0.00/6.73M [00:00<?, ?B/s] 7%|▋ | 512k/6.73M [00:00<00:01, 4.53MB/s] 20%|██ | 1.38M/6.73M [00:00<00:00, 6.83MB/s] 35%|███▌ | 2.38M/6.73M [00:00<00:00, 8.12MB/s] 50%|█████ | 3.38M/6.73M [00:00<00:00, 8.86MB/s] 69%|██████▊ | 4.62M/6.73M [00:00<00:00, 9.90MB/s] 87%|████████▋ | 5.88M/6.73M [00:00<00:00, 10.9MB/s] 100%|██████████| 6.73M/6.73M [00:00<00:00, 9.95MB/s]
Ultralytics YOLOv8.1.8 🚀 Python-3.11.2 torch-2.3.0.dev20240206+rocm5.7 CPU (AMD EPYC 7662 64-Core Processor)
YOLOv8n-seg summary (fused): 195 layers, 3404320 parameters, 0 gradients, 12.6 GFLOPs
PyTorch: starting from 'yolov8n-seg.pt' with input shape (1, 3, 480, 640) BCHW and output shape(s) ((1, 116, 6300), (1, 32, 120, 160)) (6.7 MB)
ONNX: starting export with onnx 1.15.0 opset 17...
ONNX: export success ✅ 0.7s, saved as 'yolov8n-seg.onnx' (13.2 MB)
Export complete (2.2s)
Results saved to /tmp
Predict: yolo predict task=segment model=yolov8n-seg.onnx imgsz=480,640
Validate: yolo val task=segment model=yolov8n-seg.onnx imgsz=480,640 data=coco.yaml WARNING ⚠️ non-PyTorch val requires square images, 'imgsz=[480, 640]' will not work. Use export 'imgsz=640' if val is required.
Visualize: https://netron.app
{'images': (1, 3, 480, 640)}
0: op Conv shape [(1, 3, 480, 640), (16, 3, 3, 3), (16,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (2, 2)}
1: op Sigmoid shape [(1, 16, 240, 320)] opt {}
2: op Mul shape [(1, 16, 240, 320), (1, 16, 240, 320)] opt {}
3: op Conv shape [(1, 16, 240, 320), (32, 16, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (2, 2)}
4: op Sigmoid shape [(1, 32, 120, 160)] opt {}
5: op Mul shape [(1, 32, 120, 160), (1, 32, 120, 160)] opt {}
6: op Conv shape [(1, 32, 120, 160), (32, 32, 1, 1), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
7: op Sigmoid shape [(1, 32, 120, 160)] opt {}
8: op Mul shape [(1, 32, 120, 160), (1, 32, 120, 160)] opt {}
9: op Constant shape [] opt {'value': <Tensor <LB GPU (2,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
10: op Split shape [(1, 32, 120, 160), (2,)] opt {'axis': 1}
11: op Conv shape [(1, 16, 120, 160), (16, 16, 3, 3), (16,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
12: op Sigmoid shape [(1, 16, 120, 160)] opt {}
13: op Mul shape [(1, 16, 120, 160), (1, 16, 120, 160)] opt {}
14: op Conv shape [(1, 16, 120, 160), (16, 16, 3, 3), (16,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
15: op Sigmoid shape [(1, 16, 120, 160)] opt {}
16: op Mul shape [(1, 16, 120, 160), (1, 16, 120, 160)] opt {}
17: op Add shape [(1, 16, 120, 160), (1, 16, 120, 160)] opt {}
18: op Concat shape [(1, 16, 120, 160), (1, 16, 120, 160), (1, 16, 120, 160)] opt {'axis': 1}
19: op Conv shape [(1, 48, 120, 160), (32, 48, 1, 1), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
20: op Sigmoid shape [(1, 32, 120, 160)] opt {}
21: op Mul shape [(1, 32, 120, 160), (1, 32, 120, 160)] opt {}
22: op Conv shape [(1, 32, 120, 160), (64, 32, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (2, 2)}
23: op Sigmoid shape [(1, 64, 60, 80)] opt {}
24: op Mul shape [(1, 64, 60, 80), (1, 64, 60, 80)] opt {}
25: op Conv shape [(1, 64, 60, 80), (64, 64, 1, 1), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
26: op Sigmoid shape [(1, 64, 60, 80)] opt {}
27: op Mul shape [(1, 64, 60, 80), (1, 64, 60, 80)] opt {}
28: op Constant shape [] opt {'value': <Tensor <LB GPU (2,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
29: op Split shape [(1, 64, 60, 80), (2,)] opt {'axis': 1}
30: op Conv shape [(1, 32, 60, 80), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
31: op Sigmoid shape [(1, 32, 60, 80)] opt {}
32: op Mul shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
33: op Conv shape [(1, 32, 60, 80), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
34: op Sigmoid shape [(1, 32, 60, 80)] opt {}
35: op Mul shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
36: op Add shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
37: op Conv shape [(1, 32, 60, 80), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
38: op Sigmoid shape [(1, 32, 60, 80)] opt {}
39: op Mul shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
40: op Conv shape [(1, 32, 60, 80), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
41: op Sigmoid shape [(1, 32, 60, 80)] opt {}
42: op Mul shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
43: op Add shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
44: op Concat shape [(1, 32, 60, 80), (1, 32, 60, 80), (1, 32, 60, 80), (1, 32, 60, 80)] opt {'axis': 1}
45: op Conv shape [(1, 128, 60, 80), (64, 128, 1, 1), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
46: op Sigmoid shape [(1, 64, 60, 80)] opt {}
47: op Mul shape [(1, 64, 60, 80), (1, 64, 60, 80)] opt {}
48: op Conv shape [(1, 64, 60, 80), (128, 64, 3, 3), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (2, 2)}
49: op Sigmoid shape [(1, 128, 30, 40)] opt {}
50: op Mul shape [(1, 128, 30, 40), (1, 128, 30, 40)] opt {}
51: op Conv shape [(1, 128, 30, 40), (128, 128, 1, 1), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
52: op Sigmoid shape [(1, 128, 30, 40)] opt {}
53: op Mul shape [(1, 128, 30, 40), (1, 128, 30, 40)] opt {}
54: op Constant shape [] opt {'value': <Tensor <LB GPU (2,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
55: op Split shape [(1, 128, 30, 40), (2,)] opt {'axis': 1}
56: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
57: op Sigmoid shape [(1, 64, 30, 40)] opt {}
58: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
59: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
60: op Sigmoid shape [(1, 64, 30, 40)] opt {}
61: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
62: op Add shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
63: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
64: op Sigmoid shape [(1, 64, 30, 40)] opt {}
65: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
66: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
67: op Sigmoid shape [(1, 64, 30, 40)] opt {}
68: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
69: op Add shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
70: op Concat shape [(1, 64, 30, 40), (1, 64, 30, 40), (1, 64, 30, 40), (1, 64, 30, 40)] opt {'axis': 1}
71: op Conv shape [(1, 256, 30, 40), (128, 256, 1, 1), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
72: op Sigmoid shape [(1, 128, 30, 40)] opt {}
73: op Mul shape [(1, 128, 30, 40), (1, 128, 30, 40)] opt {}
74: op Conv shape [(1, 128, 30, 40), (256, 128, 3, 3), (256,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (2, 2)}
75: op Sigmoid shape [(1, 256, 15, 20)] opt {}
76: op Mul shape [(1, 256, 15, 20), (1, 256, 15, 20)] opt {}
77: op Conv shape [(1, 256, 15, 20), (256, 256, 1, 1), (256,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
78: op Sigmoid shape [(1, 256, 15, 20)] opt {}
79: op Mul shape [(1, 256, 15, 20), (1, 256, 15, 20)] opt {}
80: op Constant shape [] opt {'value': <Tensor <LB GPU (2,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
81: op Split shape [(1, 256, 15, 20), (2,)] opt {'axis': 1}
82: op Conv shape [(1, 128, 15, 20), (128, 128, 3, 3), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
83: op Sigmoid shape [(1, 128, 15, 20)] opt {}
84: op Mul shape [(1, 128, 15, 20), (1, 128, 15, 20)] opt {}
85: op Conv shape [(1, 128, 15, 20), (128, 128, 3, 3), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
86: op Sigmoid shape [(1, 128, 15, 20)] opt {}
87: op Mul shape [(1, 128, 15, 20), (1, 128, 15, 20)] opt {}
88: op Add shape [(1, 128, 15, 20), (1, 128, 15, 20)] opt {}
89: op Concat shape [(1, 128, 15, 20), (1, 128, 15, 20), (1, 128, 15, 20)] opt {'axis': 1}
90: op Conv shape [(1, 384, 15, 20), (256, 384, 1, 1), (256,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
91: op Sigmoid shape [(1, 256, 15, 20)] opt {}
92: op Mul shape [(1, 256, 15, 20), (1, 256, 15, 20)] opt {}
93: op Conv shape [(1, 256, 15, 20), (128, 256, 1, 1), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
94: op Sigmoid shape [(1, 128, 15, 20)] opt {}
95: op Mul shape [(1, 128, 15, 20), (1, 128, 15, 20)] opt {}
96: op MaxPool shape [(1, 128, 15, 20)] opt {'ceil_mode': 0, 'dilations': (1, 1), 'kernel_shape': (5, 5), 'pads': (2, 2, 2, 2), 'strides': (1, 1)}
97: op MaxPool shape [(1, 128, 15, 20)] opt {'ceil_mode': 0, 'dilations': (1, 1), 'kernel_shape': (5, 5), 'pads': (2, 2, 2, 2), 'strides': (1, 1)}
98: op MaxPool shape [(1, 128, 15, 20)] opt {'ceil_mode': 0, 'dilations': (1, 1), 'kernel_shape': (5, 5), 'pads': (2, 2, 2, 2), 'strides': (1, 1)}
99: op Concat shape [(1, 128, 15, 20), (1, 128, 15, 20), (1, 128, 15, 20), (1, 128, 15, 20)] opt {'axis': 1}
100: op Conv shape [(1, 512, 15, 20), (256, 512, 1, 1), (256,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
101: op Sigmoid shape [(1, 256, 15, 20)] opt {}
102: op Mul shape [(1, 256, 15, 20), (1, 256, 15, 20)] opt {}
103: op Constant shape [] opt {'value': <Tensor <LB GPU (4,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
104: op Resize shape [(1, 256, 15, 20), None, (4,)] opt {'coordinate_transformation_mode': 'asymmetric', 'cubic_coeff_a': -0.75, 'mode': 'nearest', 'nearest_mode': 'floor'}
105: op Concat shape [(1, 256, 30, 40), (1, 128, 30, 40)] opt {'axis': 1}
106: op Conv shape [(1, 384, 30, 40), (128, 384, 1, 1), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
107: op Sigmoid shape [(1, 128, 30, 40)] opt {}
108: op Mul shape [(1, 128, 30, 40), (1, 128, 30, 40)] opt {}
109: op Split shape [(1, 128, 30, 40), (2,)] opt {'axis': 1}
110: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
111: op Sigmoid shape [(1, 64, 30, 40)] opt {}
112: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
113: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
114: op Sigmoid shape [(1, 64, 30, 40)] opt {}
115: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
116: op Concat shape [(1, 64, 30, 40), (1, 64, 30, 40), (1, 64, 30, 40)] opt {'axis': 1}
117: op Conv shape [(1, 192, 30, 40), (128, 192, 1, 1), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
118: op Sigmoid shape [(1, 128, 30, 40)] opt {}
119: op Mul shape [(1, 128, 30, 40), (1, 128, 30, 40)] opt {}
120: op Constant shape [] opt {'value': <Tensor <LB GPU (4,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
121: op Resize shape [(1, 128, 30, 40), None, (4,)] opt {'coordinate_transformation_mode': 'asymmetric', 'cubic_coeff_a': -0.75, 'mode': 'nearest', 'nearest_mode': 'floor'}
122: op Concat shape [(1, 128, 60, 80), (1, 64, 60, 80)] opt {'axis': 1}
123: op Conv shape [(1, 192, 60, 80), (64, 192, 1, 1), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
124: op Sigmoid shape [(1, 64, 60, 80)] opt {}
125: op Mul shape [(1, 64, 60, 80), (1, 64, 60, 80)] opt {}
126: op Split shape [(1, 64, 60, 80), (2,)] opt {'axis': 1}
127: op Conv shape [(1, 32, 60, 80), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
128: op Sigmoid shape [(1, 32, 60, 80)] opt {}
129: op Mul shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
130: op Conv shape [(1, 32, 60, 80), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
131: op Sigmoid shape [(1, 32, 60, 80)] opt {}
132: op Mul shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
133: op Concat shape [(1, 32, 60, 80), (1, 32, 60, 80), (1, 32, 60, 80)] opt {'axis': 1}
134: op Conv shape [(1, 96, 60, 80), (64, 96, 1, 1), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
135: op Sigmoid shape [(1, 64, 60, 80)] opt {}
136: op Mul shape [(1, 64, 60, 80), (1, 64, 60, 80)] opt {}
137: op Conv shape [(1, 64, 60, 80), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (2, 2)}
138: op Sigmoid shape [(1, 64, 30, 40)] opt {}
139: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
140: op Concat shape [(1, 64, 30, 40), (1, 128, 30, 40)] opt {'axis': 1}
141: op Conv shape [(1, 192, 30, 40), (128, 192, 1, 1), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
142: op Sigmoid shape [(1, 128, 30, 40)] opt {}
143: op Mul shape [(1, 128, 30, 40), (1, 128, 30, 40)] opt {}
144: op Split shape [(1, 128, 30, 40), (2,)] opt {'axis': 1}
145: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
146: op Sigmoid shape [(1, 64, 30, 40)] opt {}
147: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
148: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
149: op Sigmoid shape [(1, 64, 30, 40)] opt {}
150: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
151: op Concat shape [(1, 64, 30, 40), (1, 64, 30, 40), (1, 64, 30, 40)] opt {'axis': 1}
152: op Conv shape [(1, 192, 30, 40), (128, 192, 1, 1), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
153: op Sigmoid shape [(1, 128, 30, 40)] opt {}
154: op Mul shape [(1, 128, 30, 40), (1, 128, 30, 40)] opt {}
155: op Conv shape [(1, 128, 30, 40), (128, 128, 3, 3), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (2, 2)}
156: op Sigmoid shape [(1, 128, 15, 20)] opt {}
157: op Mul shape [(1, 128, 15, 20), (1, 128, 15, 20)] opt {}
158: op Concat shape [(1, 128, 15, 20), (1, 256, 15, 20)] opt {'axis': 1}
159: op Conv shape [(1, 384, 15, 20), (256, 384, 1, 1), (256,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
160: op Sigmoid shape [(1, 256, 15, 20)] opt {}
161: op Mul shape [(1, 256, 15, 20), (1, 256, 15, 20)] opt {}
162: op Split shape [(1, 256, 15, 20), (2,)] opt {'axis': 1}
163: op Conv shape [(1, 128, 15, 20), (128, 128, 3, 3), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
164: op Sigmoid shape [(1, 128, 15, 20)] opt {}
165: op Mul shape [(1, 128, 15, 20), (1, 128, 15, 20)] opt {}
166: op Conv shape [(1, 128, 15, 20), (128, 128, 3, 3), (128,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
167: op Sigmoid shape [(1, 128, 15, 20)] opt {}
168: op Mul shape [(1, 128, 15, 20), (1, 128, 15, 20)] opt {}
169: op Concat shape [(1, 128, 15, 20), (1, 128, 15, 20), (1, 128, 15, 20)] opt {'axis': 1}
170: op Conv shape [(1, 384, 15, 20), (256, 384, 1, 1), (256,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
171: op Sigmoid shape [(1, 256, 15, 20)] opt {}
172: op Mul shape [(1, 256, 15, 20), (1, 256, 15, 20)] opt {}
173: op Conv shape [(1, 64, 60, 80), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
174: op Sigmoid shape [(1, 64, 60, 80)] opt {}
175: op Mul shape [(1, 64, 60, 80), (1, 64, 60, 80)] opt {}
176: op ConvTranspose shape [(1, 64, 60, 80), (64, 64, 2, 2), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (2, 2), 'pads': (0, 0, 0, 0), 'strides': (2, 2)}
177: op Conv shape [(1, 64, 120, 160), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
178: op Sigmoid shape [(1, 64, 120, 160)] opt {}
179: op Mul shape [(1, 64, 120, 160), (1, 64, 120, 160)] opt {}
180: op Conv shape [(1, 64, 120, 160), (32, 64, 1, 1), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
181: op Sigmoid shape [(1, 32, 120, 160)] opt {}
182: op Mul shape [(1, 32, 120, 160), (1, 32, 120, 160)] opt {}
183: op Conv shape [(1, 64, 60, 80), (32, 64, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
184: op Sigmoid shape [(1, 32, 60, 80)] opt {}
185: op Mul shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
186: op Conv shape [(1, 32, 60, 80), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
187: op Sigmoid shape [(1, 32, 60, 80)] opt {}
188: op Mul shape [(1, 32, 60, 80), (1, 32, 60, 80)] opt {}
189: op Conv shape [(1, 32, 60, 80), (32, 32, 1, 1), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
190: op Constant shape [] opt {'value': <Tensor <LB GPU (3,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
191: op Constant shape [] opt {'value': <Tensor <LB GPU (3,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
192: op Constant shape [] opt {'value': <Tensor <LB GPU (3,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
193: op Reshape shape [(1, 32, 60, 80), (3,)] opt {'allowzero': 0}
194: op Conv shape [(1, 128, 30, 40), (32, 128, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
195: op Sigmoid shape [(1, 32, 30, 40)] opt {}
196: op Mul shape [(1, 32, 30, 40), (1, 32, 30, 40)] opt {}
197: op Conv shape [(1, 32, 30, 40), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
198: op Sigmoid shape [(1, 32, 30, 40)] opt {}
199: op Mul shape [(1, 32, 30, 40), (1, 32, 30, 40)] opt {}
200: op Conv shape [(1, 32, 30, 40), (32, 32, 1, 1), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
201: op Reshape shape [(1, 32, 30, 40), (3,)] opt {'allowzero': 0}
202: op Conv shape [(1, 256, 15, 20), (32, 256, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
203: op Sigmoid shape [(1, 32, 15, 20)] opt {}
204: op Mul shape [(1, 32, 15, 20), (1, 32, 15, 20)] opt {}
205: op Conv shape [(1, 32, 15, 20), (32, 32, 3, 3), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
206: op Sigmoid shape [(1, 32, 15, 20)] opt {}
207: op Mul shape [(1, 32, 15, 20), (1, 32, 15, 20)] opt {}
208: op Conv shape [(1, 32, 15, 20), (32, 32, 1, 1), (32,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
209: op Reshape shape [(1, 32, 15, 20), (3,)] opt {'allowzero': 0}
210: op Concat shape [(1, 32, 4800), (1, 32, 1200), (1, 32, 300)] opt {'axis': 2}
211: op Conv shape [(1, 64, 60, 80), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
212: op Sigmoid shape [(1, 64, 60, 80)] opt {}
213: op Mul shape [(1, 64, 60, 80), (1, 64, 60, 80)] opt {}
214: op Conv shape [(1, 64, 60, 80), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
215: op Sigmoid shape [(1, 64, 60, 80)] opt {}
216: op Mul shape [(1, 64, 60, 80), (1, 64, 60, 80)] opt {}
217: op Conv shape [(1, 64, 60, 80), (64, 64, 1, 1), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
218: op Conv shape [(1, 64, 60, 80), (80, 64, 3, 3), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
219: op Sigmoid shape [(1, 80, 60, 80)] opt {}
220: op Mul shape [(1, 80, 60, 80), (1, 80, 60, 80)] opt {}
221: op Conv shape [(1, 80, 60, 80), (80, 80, 3, 3), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
222: op Sigmoid shape [(1, 80, 60, 80)] opt {}
223: op Mul shape [(1, 80, 60, 80), (1, 80, 60, 80)] opt {}
224: op Conv shape [(1, 80, 60, 80), (80, 80, 1, 1), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
225: op Concat shape [(1, 64, 60, 80), (1, 80, 60, 80)] opt {'axis': 1}
226: op Conv shape [(1, 128, 30, 40), (64, 128, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
227: op Sigmoid shape [(1, 64, 30, 40)] opt {}
228: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
229: op Conv shape [(1, 64, 30, 40), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
230: op Sigmoid shape [(1, 64, 30, 40)] opt {}
231: op Mul shape [(1, 64, 30, 40), (1, 64, 30, 40)] opt {}
232: op Conv shape [(1, 64, 30, 40), (64, 64, 1, 1), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
233: op Conv shape [(1, 128, 30, 40), (80, 128, 3, 3), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
234: op Sigmoid shape [(1, 80, 30, 40)] opt {}
235: op Mul shape [(1, 80, 30, 40), (1, 80, 30, 40)] opt {}
236: op Conv shape [(1, 80, 30, 40), (80, 80, 3, 3), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
237: op Sigmoid shape [(1, 80, 30, 40)] opt {}
238: op Mul shape [(1, 80, 30, 40), (1, 80, 30, 40)] opt {}
239: op Conv shape [(1, 80, 30, 40), (80, 80, 1, 1), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
240: op Concat shape [(1, 64, 30, 40), (1, 80, 30, 40)] opt {'axis': 1}
241: op Conv shape [(1, 256, 15, 20), (64, 256, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
242: op Sigmoid shape [(1, 64, 15, 20)] opt {}
243: op Mul shape [(1, 64, 15, 20), (1, 64, 15, 20)] opt {}
244: op Conv shape [(1, 64, 15, 20), (64, 64, 3, 3), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
245: op Sigmoid shape [(1, 64, 15, 20)] opt {}
246: op Mul shape [(1, 64, 15, 20), (1, 64, 15, 20)] opt {}
247: op Conv shape [(1, 64, 15, 20), (64, 64, 1, 1), (64,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
248: op Conv shape [(1, 256, 15, 20), (80, 256, 3, 3), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
249: op Sigmoid shape [(1, 80, 15, 20)] opt {}
250: op Mul shape [(1, 80, 15, 20), (1, 80, 15, 20)] opt {}
251: op Conv shape [(1, 80, 15, 20), (80, 80, 3, 3), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (3, 3), 'pads': (1, 1, 1, 1), 'strides': (1, 1)}
252: op Sigmoid shape [(1, 80, 15, 20)] opt {}
253: op Mul shape [(1, 80, 15, 20), (1, 80, 15, 20)] opt {}
254: op Conv shape [(1, 80, 15, 20), (80, 80, 1, 1), (80,)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
255: op Concat shape [(1, 64, 15, 20), (1, 80, 15, 20)] opt {'axis': 1}
256: op Constant shape [] opt {'value': <Tensor <LB GPU (3,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
257: op Constant shape [] opt {'value': <Tensor <LB GPU (3,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
258: op Constant shape [] opt {'value': <Tensor <LB GPU (3,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
259: op Reshape shape [(1, 144, 60, 80), (3,)] opt {'allowzero': 0}
260: op Reshape shape [(1, 144, 30, 40), (3,)] opt {'allowzero': 0}
261: op Reshape shape [(1, 144, 15, 20), (3,)] opt {'allowzero': 0}
262: op Concat shape [(1, 144, 4800), (1, 144, 1200), (1, 144, 300)] opt {'axis': 2}
263: op Constant shape [] opt {'value': <Tensor <LB GPU (2,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
264: op Split shape [(1, 144, 6300), (2,)] opt {'axis': 1}
265: op Constant shape [] opt {'value': <Tensor <LB GPU (4,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
266: op Reshape shape [(1, 64, 6300), (4,)] opt {'allowzero': 0}
267: op Transpose shape [(1, 4, 16, 6300)] opt {'perm': (0, 2, 1, 3)}
268: op Softmax shape [(1, 16, 4, 6300)] opt {'axis': 1}
269: op Conv shape [(1, 16, 4, 6300), (1, 16, 1, 1)] opt {'dilations': (1, 1), 'group': 1, 'kernel_shape': (1, 1), 'pads': (0, 0, 0, 0), 'strides': (1, 1)}
270: op Constant shape [] opt {'value': <Tensor <LB GPU (3,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
271: op Reshape shape [(1, 1, 4, 6300), (3,)] opt {'allowzero': 0}
272: op Shape shape [(1, 4, 6300)] opt {}
273: op Constant shape [] opt {'value': <Tensor <LB GPU (1,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
274: op Gather shape [(3,), (1,)] opt {'axis': 0}
275: op Constant shape [] opt {'value': <Tensor <LB GPU (1,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
276: op Constant shape [] opt {'value': <Tensor <LB GPU (1,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
277: op Add shape [(1,), (1,)] opt {}
278: op Constant shape [] opt {'value': <Tensor <LB GPU (1,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
279: op Div shape [(1,), (1,)] opt {}
280: op Constant shape [] opt {'value': <Tensor <LB GPU (1,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
281: op Mul shape [(1,), (1,)] opt {}
282: op Slice shape [(1, 4, 6300), (1,), (1,), (1,)] opt {}
283: op Constant shape [] opt {'value': <Tensor <LB GPU (1,) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
284: op Mul shape [(1,), (1,)] opt {}
285: op Slice shape [(1, 4, 6300), (1,), (1,), (1,)] opt {}
286: op Constant shape [] opt {'value': <Tensor <LB GPU (1, 2, 6300) contig:True (<LoadOps.COPY: 3>, None)> on GPU with grad None>}
287: op Sub shape [(1, 2, 6300), (1, 3, 6300)] opt {}
Traceback (most recent call last):
File "/home/jebba/devel/tinygrad/tinygrad/examples/yolov8-onnx.py", line 18, in <module>
run_onnx({"images": Tensor.zeros(1,3,480,640)}, debug=True)
File "/home/jebba/devel/tinygrad/tinygrad/extra/onnx.py", line 211, in run_onnx
ret = real_fxn(*inp, **opt)
^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/extra/onnx_ops.py", line 18, in Sub
def Sub(x: Union[Tensor, Any], other: Tensor): return x - other # some test has input as int
~~^~~~~~~
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/tensor.py", line 858, in __sub__
def __sub__(self, x) -> Tensor: return self.sub(x)
^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/tensor.py", line 812, in sub
return mlops.Sub.apply(*self._broadcasted(x, reverse)) if x.__class__ is Tensor or x else (-self if reverse else self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/tensor.py", line 800, in _broadcasted
return x.expand(broadcasted_shape), y.expand(broadcasted_shape)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/tensor.py", line 309, in expand
return mlops.Expand.apply(self, shape=new_shape) if new_shape != self.shape else self
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/tensor.py", line 34, in apply
ret.lazydata, ret.requires_grad, ret.grad = ctx.forward(*[t.lazydata for t in x], **kwargs), ctx.requires_grad, None
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/mlops.py", line 168, in forward
return x.expand(shape)
^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/lazy.py", line 147, in expand
def expand(self, arg:Tuple[sint, ...]): return self._view(self.st.expand(arg))
^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/shape/shapetracker.py", line 180, in expand
def expand(self, new_shape: Tuple[sint, ...]) -> ShapeTracker: return ShapeTracker(self.views[0:-1] + (self.views[-1].expand(new_shape), ))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jebba/devel/tinygrad/tinygrad/tinygrad/shape/view.py", line 156, in expand
assert all((s == x or (s == 1 and st == 0)) for s,x,st in zip(self.shape, new_shape, self.strides)), f"can't expand {self.shape} into {new_shape}"
AssertionError: can't expand (1, 2, 6300) into (1, 3, 6300)

1
docs/_source/_static/_output/tinygrad/yolov8.py.txt 100644
View File

@ -0,0 +1 @@
Error: Image URL or path not provided.

42
docs/_source/locale/en/LC_MESSAGES/output.po
View File

@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: tinyrocs 0\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2024-02-06 11:49-0700\n"
"POT-Creation-Date: 2024-02-06 12:20-0700\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language: en\n"
@ -109,8 +109,8 @@ msgstr ""
#: ../../../_source/output.rst:86
msgid ""
"Note, while these examples were running, builds were also running, "
"hitting 128 processors, so these examples aren't benchmarks."
"Note, while these examples were running, builds were also running, hitting "
"128 processors, so these examples aren't benchmarks."
msgstr ""
#: ../../../_source/output.rst:90
@ -197,6 +197,38 @@ msgstr ""
msgid "``python examples/train_efficientnet.py``"
msgstr ""
#~ msgid "System _output."
#~ msgstr ""
#: ../../../_source/output.rst:216
msgid "``python examples/train_resnet.py``"
msgstr ""
#: ../../../_source/output.rst:222
msgid "``python examples/transformer.py``"
msgstr ""
#: ../../../_source/output.rst:228
msgid "``python examples/vgg7.py``"
msgstr ""
#: ../../../_source/output.rst:234
msgid "``python examples/vit.py``"
msgstr ""
#: ../../../_source/output.rst:240
msgid "``python examples/vits.py``"
msgstr ""
#: ../../../_source/output.rst:246
msgid "``python examples/whisper.py``"
msgstr ""
#: ../../../_source/output.rst:252
msgid "``python examples/yolov3.py``"
msgstr ""
#: ../../../_source/output.rst:258
msgid "``python examples/yolov8-onnx.py``"
msgstr ""
#: ../../../_source/output.rst:264
msgid "``python examples/yolov8.py``"
msgstr ""

54
docs/_source/output.rst
View File

@ -212,3 +212,57 @@ Note, while these examples were running, builds were also running, hitting
.. literalinclude:: _static/_output/tinygrad/train_efficientnet.py.txt
:language: output
``python examples/train_resnet.py``
-----------------------------------
.. literalinclude:: _static/_output/tinygrad/train_resnet.py.txt
:language: output
``python examples/transformer.py``
----------------------------------
.. literalinclude:: _static/_output/tinygrad/transformer.py.txt
:language: output
``python examples/vgg7.py``
---------------------------
.. literalinclude:: _static/_output/tinygrad/vgg7.py.txt
:language: output
``python examples/vit.py``
--------------------------
.. literalinclude:: _static/_output/tinygrad/vit.py.txt
:language: output
``python examples/vits.py``
---------------------------
.. literalinclude:: _static/_output/tinygrad/vits.py.txt
:language: output
``python examples/whisper.py``
------------------------------
.. literalinclude:: _static/_output/tinygrad/whisper.py.txt
:language: output
``python examples/yolov3.py``
-----------------------------
.. literalinclude:: _static/_output/tinygrad/yolov3.py.txt
:language: output
``python examples/yolov8-onnx.py``
----------------------------------
.. literalinclude:: _static/_output/tinygrad/yolov8-onnx.py.txt
:language: output
``python examples/yolov8.py``
-----------------------------
.. literalinclude:: _static/_output/tinygrad/yolov8.py.txt
:language: output