For Ubuntu, install the latest nvidia drivers from the repo and install Cuda Toolkit separately without the drivers.
You may also want to install cuDnn
For tensorflow and pytorch, you may need to add
LD_LIBRARY_PATH=/usr/local/cuda/lib64 to your environment variables.
You can also do this in PyCharm.