Google Colab: Difference between revisions
| Line 12: | Line 12: | ||
* Run <code>curl -r 0-4999999999 [url] -o part1</code> to download ~50GB your Google Drive | * Run <code>curl -r 0-4999999999 [url] -o part1</code> to download ~50GB your Google Drive | ||
* Wait ~10 minutes for the cache to clear and files to copy over | * Wait ~10 minutes for the cache to clear and files to copy over | ||
* Do <code>curl -r 5000000000-9999999999 [url] -o part2<code> | * Do <code>curl -r 5000000000-9999999999 [url] -o part2</code> | ||
* Repeat until you have the whole >50GB file | * Repeat until you have the whole >50GB file | ||
* . | * . | ||
Revision as of 15:33, 9 June 2020
Google Colab is a free online Jupyter notebook you can access for small ML training exercises.
It supports Python and R.
Google Drive
Colab supports mounting Google Drive.
However, files are cached locally on the VM which only has ~100GB.
This means you cannot directly write (e.g. via curl) files over 100GB to your Google Drive folder.
There are some tricks you can try though:
curl
- Run
curl -r 0-4999999999 [url] -o part1to download ~50GB your Google Drive - Wait ~10 minutes for the cache to clear and files to copy over
- Do
curl -r 5000000000-9999999999 [url] -o part2 - Repeat until you have the whole >50GB file
- .
Colab Pro
Claims faster GPUs, longer runtimes, and more memory.
However nothing is guaranteed and VMs are still preemptible.
Runtimes are limited to 24 hours as opposed to 12 hours for free users.