TensorFlow is the famous machine learning library by Google
- Install CUDA and CuDNN
- Create a conda environment with python 3.7
- You can also just create a tensorflow environment using conda
conda create -n my_env tensorflow
- Install with pip
pip install tensorflow
pip install tensorflow-gpu==1.15
Here we'll cover usage using TensorFlow 2 which has eager execution.
This is using the Keras API in tensorflow.keras.
The general pipeline using Keras is:
- Define a model, typically using tf.keras.Sequential
- Here you pass in your optimizer, loss function, and metrics.
- Train your model by calling
- Here you pass in your training data, batch size, number of epochs, and training callbacks
- For more information about callbacks, see Keras custom callbacks.
After training, you can use your model by calling
An alternative way to define a model is by extending the Model class:
- Write a python class which extends
- Implement a forward pass in the
Custom Training Loop
While you can train using
model.fit, using your own custom training loop is much more flexable and easier to understand.
You can write your own training loop by doing the following:
my_model= keras.Sequential([ keras.layers.Dense(400, input_shape=400, activation='relu'), keras.layers.Dense(400, activation='relu'), keras.layers.Dense(400, activation='relu'), keras.layers.Dense(400, activation='relu'), keras.layers.Dense(400, activation='relu'), keras.layers.Dense(2) ]) optimizer = keras.optimizers.SGD(learning_rate=1e-3) training_loss =  validation_loss =  for epoch in range(100): print('Start of epoch %d' % (epoch,)) for step, (x_batch_train, y_batch_train) in enumerate(train_dataset): with tf.GradientTape() as tape: guess = my_model(x_batch_train) loss_value = my_custom_loss(y_batch_train, guess) # Use the gradient tape to automatically retrieve # the gradients of the trainable variables with respect to the loss. grads = tape.gradient(loss_value, my_model.trainable_weights) # Run one step of gradient descent by updating # the value of the variables to minimize the loss. optimizer.apply_gradients(zip(grads, my_model.trainable_weights)) # Log every 200 batches. if step % 200 == 0: print('Training loss at step %s: %s' % (step, float(loss_value))) training_loss.append(loss_value) guess_validation = model(x_validation) validation_loss.append(my_custom_loss(y_validation, guess_validation))
Save and Load Models
You can extract the training loss from the events file in tensorflow.