* Added standalone CLIP tokenizer.
* Fixed empty phrase.
* Truncating long prompts.
* Keeping two slots for the start and end token.
* Fixed empty phrase.
* Using tokenizer for empty phrase.
* Typo.
* added resnets
* fix minor
* fix minor
* resnet in models
* added resnet test
* added resnet train test
* added linear, conv2d nn tests
* fix minor in extra/training
* resnet in models
* fix minor
* fix tolerance for linear in nn test
* fix eval, this causes cpu and gpu UT failing
* revert transformer test
* fix minor for CPU test
* improved model get_params for sequential layer
* fix minor for params counting
* commented broken ops tests
* improved train for resnet
* use isinstance, some optimizations & whitespace removal
* revert whitespace changes
* revert more whitespace
* some more cleanup
* revert fstring (not a fan of the {{}})
* fix typo
* fix typo
* vgg7 implementation - not the best, but it works
* VGG7 implementation: Spread nansbane to deter NaNs, maybe improved training experience
* VGG7 implementation: Fix training, for real this time
Results actually attempt to approximate the input
* VGG7 implementation: Sample probability management
* Some progress on yolov3
* Removed some debugging comments… Also, the forward pass eats all RAM for some reason
* forward pass almost runs
* forward pass runs almost
* forward pass runs, now we gotta load the weights
* loading weights works
* fetches config and weights
* everything kind of works, postprocessing of output still needs to be implemented, temp_process_results kind of works, but its kind of terrible, and not how things should be done
* some changes
* fixed some bugs in the forward pass and load_weights function, now outputs more correct values, however some values are still loaded incorrectly
* Something is wrong with the forward pass, Conv2d tests added
* forward pass almost outputs correct values, gotta fix one more thign
* yolo works
* some final changes
* reverting changes
* removed dataloader
* fixed some indentation
* comment out failing test, somehow it fails CI even though it passes on my computer…
* fixed wrong probabilities
* added webcam option to YOLO, now just need to add bounding boxes and speed it up
* some progress towards adding bounding boxes
* trying to speed up yolo layer on GPU, still faster on CPU but with 30GB ram usage
* Faster inference times, bounding boxes added correctly, webcam works, but is slow, and there is a memory leak when running on CPU... Also added tinygrads output on the classic dog image
* removed some debugging print statements
* updated result image
* something weird is going on, mean op on GPU tensor randomly faults, copying a tensor from GPU->CPU takes 10+ seconds…
* Improved __getitem__
* Updated
* Updated __getitem__
* Linebreaks
* Maybe this works?
* Added MNIST locally, tests run now
* Some progress on yolov3
* Removed some debugging comments… Also, the forward pass eats all RAM for some reason
* forward pass almost runs
* forward pass runs almost
* forward pass runs, now we gotta load the weights
* loading weights works
* fetches config and weights
* everything kind of works, postprocessing of output still needs to be implemented, temp_process_results kind of works, but its kind of terrible, and not how things should be done
* some changes
* fixed some bugs in the forward pass and load_weights function, now outputs more correct values, however some values are still loaded incorrectly
* Something is wrong with the forward pass, Conv2d tests added
* forward pass almost outputs correct values, gotta fix one more thign
* yolo works
* some final changes
* reverting changes
* removed dataloader
* fixed some indentation
* comment out failing test, somehow it fails CI even though it passes on my computer…
* fixed wrong probabilities
* added webcam option to YOLO, now just need to add bounding boxes and speed it up
* some progress towards adding bounding boxes
* trying to speed up yolo layer on GPU, still faster on CPU but with 30GB ram usage
* Faster inference times, bounding boxes added correctly, webcam works, but is slow, and there is a memory leak when running on CPU... Also added tinygrads output on the classic dog image
* removed some debugging print statements
* updated result image
* something weird is going on, mean op on GPU tensor randomly faults, copying a tensor from GPU->CPU takes 10+ seconds…
* 2serious
* load/save
* fixing GPU
* added DEBUG
* needs BatchNorm or doesn't learn anything
* old file not needed
* added conv biases
* added extra/training.py and checkpoint
* assert in test only
* save
* padding
* num_classes
* checkpoint
* checkpoints for padding
* training was broken
* merge
* rotation augmentation
* more aug
* needs testing
* streamline augment, augment is fast thus bicubic
* tidying up
* transformer eval
* 2serious
* load/save
* fixing GPU
* added DEBUG
* needs BatchNorm or doesn't learn anything
* old file not needed
* added conv biases
* added extra/training.py and checkpoint
* assert in test only
* save
* padding
* num_classes
* checkpoint
* checkpoints for padding
* training was broken
* merge
* rotation augmentation
* more aug
* needs testing
* streamline augment, augment is fast thus bicubic
* tidying up
* 🎉 effort to generate mnist data with tinygrad.
* dropout added
* working gan
* minor bug fixes
* more bug fixes
* todo reg l2
* detach
* logsoftmax twice
* no of categories for efficientnet
* need layer_init_uniforn
* merge fail
* merge fail
* batchnorms
* needs work
* needs work how determine training
* pow
* needs work
* reshape was needed
* sum with axis
* sum with axis and tests
* broken
* works again
* clean up
* Update test_ops.py
* using sum
* don't always update running_stats
* space
* self
* default return running_stats
* passes test
* need to use mean
* merge
* testing
* fixing pow
* test_ops had a line dropped
* undo pow
* rebase
* to make it work locally
* definitely not working
* Conv2D GPU passes some of the tests
* Conv2D GPU passes more of the tests
* passes some tests and mnist
* removed unecessary code
* Conv2D Backpass works
* wrong test_ops.py
* white space + test backward
* ereased useless code
* removed default argument
* long lines
* works also with 4 channel .png files
* commenting out
* track
* pygame is fine, cv2 can also do the trick
* retimg and copy constructor not needed
* shape is missing without copy constructor
* retimg put back
* addressing capture buffering
from https://github.com/lukemelas/EfficientNet-PyTorch/blob/master/efficientnet_pytorch/utils.py
```
blocks_args = [
'r1_k3_s11_e1_i32_o16_se0.25',
'r2_k3_s22_e6_i16_o24_se0.25',
'r2_k5_s22_e6_i24_o40_se0.25',
'r3_k3_s22_e6_i40_o80_se0.25',
'r3_k5_s11_e6_i80_o112_se0.25',
'r4_k5_s22_e6_i112_o192_se0.25',
'r1_k3_s11_e6_i192_o320_se0.25',
]
```
now it's a persian cat.
* streamlined numerical_jacobian
* Got rid of the g loop in Conv2D.forward
* ereased stupid line
* nothing
* no loops in Conv2D forward
* Conv2D backprop improved
* stupid things in examples
* alternative to einsum
* Conv2D backward einsum alternative
* tidying up
* tidied up
* no ravel
* got rid of print
* Update efficientnet.py
* Update efficientnet.py
* Update efficientnet.py
* only tensordot
* 255.0
* whitespace
* aspect ratio error in efficientnet
* noprint
Co-authored-by: Marcel Bischoff <marcel@Marcels-iMac.local>
George Hotz