Ukwandisa isantya seTensorFlow kwi-50% nge-TPU Tricks

Iinkcukacha Overview Ukucinga Ukulungiswa kwe-TPU" Manual Ukupakisha Strategy Ukudibanisa Ukucaciswa kwi-TPUs Ukucaciswa kwe-Keras Model Yenza i-DataSet Ukuqeqesha imodeli usebenzisa i-Keras API ye-high-level Ukuqeqesha imodeli usebenzisa i-custom training loop Ukuphucula ukusebenza ngeengxaki ezininzi ngaphakathi tf.function Ukucaciswa Le nqaku ibonisa indlela yokwenza ukuqeqeshwa base kunye ne-TPU Pods, isixhobo se-TPU ezihambelana nge-high-speed network interfaces ezizodwa, kunye kunye ne-Custom Training Loops. I-Tensor Processing Units (i-TPUs) tf.keras I-TPU yi-Google's custom-developed application-specific integrated circuits (ASICs) ezisetyenziselwa ukuphucula iinkqubo zokusebenza ze-machine learning. Yintoni Yaye . I-Google Colab TPU Ubuchwepheshe Cloud I-cloud ye-TPU Ukucinga Nangona uqhagamshelane ne-Colab notebook, qinisekisa ukuba i-hardware accelerator yakho i-TPU ngokucacisa i-notebook yakho: > > > . Runtime Change runtime type Hardware accelerator TPU Izixhobo ezininzi eziluncedo, kuquka i-TensorFlow Datasets: import tensorflow as tf import os import tensorflow_datasets as tfds 2023-06-09 12:13:32.486552: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT Ukuqala TPU I-TPUs ziquka ngokuvamile abasebenzi, ezahlukileyo kwiproces ye-local esebenza kwiprogram ye-Python yomsebenzisi. Ngoko ke, kufuneka uyenze iinkonzo yokuqala ukuxhumanisa kwi-cluster ye-remote kunye nokuqinisekisa i-TPUs. Qhagamshelana Ukuba usebenze ikhowudi yakho ku-Google Compute Engine (GCE), kufuneka usebenze kwinqanaba yakho ye-Cloud TPU. I-cloud ye-TPU tpu tf.distribute.cluster_resolver.TPUClusterResolver Qaphela: Ikhodi ye-TPU ye-initialization kufuneka kuqala kwiprogram yakho. Qaphela: Ikhodi ye-TPU ye-initialization kufuneka kuqala kwiprogram yakho. resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='') tf.config.experimental_connect_to_cluster(resolver) # This is the TPU initialization code that has to be at the beginning. tf.tpu.experimental.initialize_tpu_system(resolver) print("All devices: ", tf.config.list_logical_devices('TPU')) INFO:tensorflow:Deallocate tpu buffers before initializing tpu system. 2023-06-09 12:13:34.011755: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected INFO:tensorflow:Deallocate tpu buffers before initializing tpu system. INFO:tensorflow:Initializing the TPU system: grpc://10.25.167.66:8470 INFO:tensorflow:Initializing the TPU system: grpc://10.25.167.66:8470 INFO:tensorflow:Finished initializing TPU system. INFO:tensorflow:Finished initializing TPU system. All devices: [LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:0', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:1', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:2', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:3', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:4', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:5', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:6', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:7', device_type='TPU')] Manual Ukupakisha Emva kwe-TPU ifumaneka, unako usebenzisa indawo ye-device ye-manual yokufaka i-calculation kwifowuni ye-TPU elinye: a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]]) with tf.device('/TPU:0'): c = tf.matmul(a, b) print("c device: ", c.device) print(c) c device: /job:worker/replica:0/task:0/device:TPU:0 tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32) Strategy Ukudibanisa Ngokuvamile, uqhagamshelane imodeli yakho kwi-TPU ezininzi ngokuhambelana neendaba. Ukudibanisa imodeli yakho kwi-TPU ezininzi (kwakhona kwi-GPU ezininzi okanye kwiimishini ezininzi), i-TensorFlow inikeza API. Uyakwazi ukuguqula i-distribution strategy yakho kwaye i-model iya kuqhutywa kwifowuni ye-TPU. Funda ngokugqithisileyo kwi umnikelo tf.distribute.Strategy Ukuqeqeshwa nge-TensorFlow Ukusetyenziswa i-optional ivumela ukuqeqeshwa kwe-distributed synchronous. I-TPU inikeza ukuvelisa kwakhona i-efficient all-reduce kunye nezinye iinkonzo ze-collective kwi-multiple TPU cores, ezisetyenziswa . tf.distribute.TPUStrategy TPUStrategy Ukubonisa oku, ukwakha a Ukucinga: tf.distribute.TPUStrategy strategy = tf.distribute.TPUStrategy(resolver) INFO:tensorflow:Found TPU system: INFO:tensorflow:Found TPU system: INFO:tensorflow:*** Num TPU Cores: 8 INFO:tensorflow:*** Num TPU Cores: 8 INFO:tensorflow:*** Num TPU Workers: 1 INFO:tensorflow:*** Num TPU Workers: 1 INFO:tensorflow:*** Num TPU Cores Per Worker: 8 INFO:tensorflow:*** Num TPU Cores Per Worker: 8 INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0) Ukubhalisa i-calculation ukuze ifumaneke kumadoda we-TPU yonke, unako ukuxhaswa kwi-CPU. API. Ngezantsi na umzekelo ebonakalayo zonke iingcali ezivela kwi-inputs efanayo kunye nokuveliswa kwe-matrix kwi-core ngamnye ngokufanelekileyo. I-outputs iya kuba amaxabiso ze-replicas ezininzi. Strategy.run (a, b) @tf.function def matmul_fn(x, y): z = tf.matmul(x, y) return z z = strategy.run(matmul_fn, args=(a, b)) print(z) PerReplica:{ 0: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 1: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 2: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 3: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 4: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 5: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 6: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 7: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32) } Ukucaciswa kwi-TPUs Emva kokufumana iingcebiso eziphambili, bheka isibonelo esebenzayo. Le nkqubo ibonise indlela yokusetyenziswa kwe-distribution strategy— Ukuqeqesha imodeli ye-Keras kwi-Cloud TPU. tf.distribute.TPUStrategy Ukucaciswa kwe-Keras Model Zifumaneka nge-definition ye-a I-Keras model ye-image classification kwi-MNIST dataset. Akukho nto eyahlukileyo kunokuba usebenzise ukuba usebenzise kwi-CPU okanye i-GPU. Qinisekisa ukuba ukuvelisa i-Keras model kufuneka ibe ngaphakathi kwe-CPU. , ukuze ama-variables ziyafumaneka kumadivayisi omnye ye-TPU. Ezinye iindawo ze-code ayikho kufuneka ziyafumaneka kwi-TPU Ukucinga Sequential Strategy.scope Strategy def create_model(): return tf.keras.Sequential( [tf.keras.layers.Conv2D(256, 3, activation='relu', input_shape=(28, 28, 1)), tf.keras.layers.Conv2D(256, 3, activation='relu'), tf.keras.layers.Flatten(), tf.keras.layers.Dense(256, activation='relu'), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(10)]) Yenza i-DataSet Ukusetyenziswa okufanelekileyo API kubalulekile xa usebenzisa i-Cloud TPU. Unokufunda ngakumbi malunga nenkqubo ye-dataset . tf.data.Dataset Input Pipeline Ukusebenza Guide Ukuba usebenzisa , kufuneka ukugcina zonke iifayile zeendatha ezaziwa yi-TensorFlow Ukucinga Ukuba usebenzisa , unako ukugcina idatha apho ufuna. Ukuze uthole ulwazi oluthe ngexabiso ye-TPU Nodes kunye ne-TPU VMs, nqakraza kwi Ukucaciswa Iingubo ze-TPU Dataset I-Google Cloud Storage (i-GCS) iinkcukacha Ukucinga kwe-VMs TPU System Ukwakhiwa Kwiimeko ezininzi zokusetyenziswa, kuboniswa ukuba idatha yakho format kunye nokusetyenziswa a ukuyifaka. Qhagamshelana Ukufumana iinkcukacha malunga nenkqubo yokwenza oku. It is not a hard requirement and you can use other dataset readers, such as: okanye . TFRecord tf.data.TFRecordDataset I-TFRecord kunye ne-tf.Example tutorial tf.data.FixedLengthRecordDataset tf.data.TextLineDataset Uyakwazi ukulayisha iinkcukacha ezincinane ezininzi kwi-memory usebenzisa . tf.data.Dataset.cache Nangona ifomati data esetyenziswa, kuboniswa kakhulu ukuba usebenzisa iifayile ezinkulu kwinqanaba ye-100MB. Oku kubalulekile kakhulu kule setting networked, njengoko i-overhead yokufunda ifayile kunzima kakhulu. Njengoko i-code elandelayo, kufuneka usebenzise i-Tensorflow Datasets Module ukuze ufumane i-copy ye-MNIST training and test data. Qaphela ukuba ifumaneka ukuba usebenzise i-copy enokufumaneka kwi-GCS bucket. Ukuba ungenza oku, i-TPU ayikwazi ukufikelela idatha eyenziwe. tfds.load try_gcs def get_dataset(batch_size, is_training=True): split = 'train' if is_training else 'test' dataset, info = tfds.load(name='mnist', split=split, with_info=True, as_supervised=True, try_gcs=True) # Normalize the input data. def scale(image, label): image = tf.cast(image, tf.float32) image /= 255.0 return image, label dataset = dataset.map(scale) # Only shuffle and repeat the dataset in training. The advantage of having an # infinite dataset for training is to avoid the potential last partial batch # in each epoch, so that you don't need to think about scaling the gradients # based on the actual batch size. if is_training: dataset = dataset.shuffle(10000) dataset = dataset.repeat() dataset = dataset.batch(batch_size) return dataset Ukuqeqesha imodeli usebenzisa i-Keras API ye-high-level Uyakwazi ukuqeqesha imodeli yakho nge-Keras iimveliso I-APIs. Akukho into ye-TPU-specific kule step—ukubhala ikhowudi njengoko usebenzisa i-GPU ezininzi kunye ne-GPU Kwimeko ye . Uyakwazi ukufundisa ngaphezulu kwi Ukulungiselela Model.fit Model.compile MirroredStrategy TPUStrategy Ukuqeqeshwa nge-Keras with strategy.scope(): model = create_model() model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['sparse_categorical_accuracy']) batch_size = 200 steps_per_epoch = 60000 // batch_size validation_steps = 10000 // batch_size train_dataset = get_dataset(batch_size, is_training=True) test_dataset = get_dataset(batch_size, is_training=False) model.fit(train_dataset, epochs=5, steps_per_epoch=steps_per_epoch, validation_data=test_dataset, validation_steps=validation_steps) Epoch 1/5 300/300 [==============================] - 17s 32ms/step - loss: 0.1235 - sparse_categorical_accuracy: 0.9620 - val_loss: 0.0462 - val_sparse_categorical_accuracy: 0.9856 Epoch 2/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0333 - sparse_categorical_accuracy: 0.9894 - val_loss: 0.0401 - val_sparse_categorical_accuracy: 0.9878 Epoch 3/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0186 - sparse_categorical_accuracy: 0.9938 - val_loss: 0.0352 - val_sparse_categorical_accuracy: 0.9900 Epoch 4/5 300/300 [==============================] - 7s 25ms/step - loss: 0.0127 - sparse_categorical_accuracy: 0.9957 - val_loss: 0.0482 - val_sparse_categorical_accuracy: 0.9879 Epoch 5/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0111 - sparse_categorical_accuracy: 0.9962 - val_loss: 0.0448 - val_sparse_categorical_accuracy: 0.9894 Ukunciphisa i-Python overhead kunye nokwandisa ukusebenza kwe-TPU yakho, ukufikelela kwi Iingxaki ze-Keras Kule isibonelo, kunceda ukwandisa umthamo nge-50% ngexesha: steps_per_execution Model.compile with strategy.scope(): model = create_model() model.compile(optimizer='adam', # Anything between 2 and `steps_per_epoch` could help here. steps_per_execution = 50, loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['sparse_categorical_accuracy']) model.fit(train_dataset, epochs=5, steps_per_epoch=steps_per_epoch, validation_data=test_dataset, validation_steps=validation_steps) Epoch 1/5 300/300 [==============================] - 14s 45ms/step - loss: 0.1306 - sparse_categorical_accuracy: 0.9591 - val_loss: 0.0420 - val_sparse_categorical_accuracy: 0.9863 Epoch 2/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0333 - sparse_categorical_accuracy: 0.9900 - val_loss: 0.0502 - val_sparse_categorical_accuracy: 0.9846 Epoch 3/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0193 - sparse_categorical_accuracy: 0.9936 - val_loss: 0.0406 - val_sparse_categorical_accuracy: 0.9879 Epoch 4/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0135 - sparse_categorical_accuracy: 0.9955 - val_loss: 0.0416 - val_sparse_categorical_accuracy: 0.9882 Epoch 5/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0110 - sparse_categorical_accuracy: 0.9962 - val_loss: 0.0463 - val_sparse_categorical_accuracy: 0.9882 Ukuqeqesha imodeli usebenzisa i-custom training loop Uyakwazi ukuvelisa kunye nokulungisa imodeli yakho usebenzisa iimveliso APIs ngqo. Ungasebenzisa i I-Fire to distribute it umzekelo we-dataset. Qaphela ukuba kwi-example ngezantsi i-batch size ifumaneka kwi- i-per-replica batch size kunokuba yi-batch size jikelele. Ukuze ufunde ngaphezulu, nqakraza i Ukulungiselela tf.function tf.distribute Strategy.experimental_distribute_datasets_from_function tf.data.Dataset Dataset Custom Ukuqeqeshwa nge tf.distribute.Strategy Okokuqala, ukwakha model, datasets kunye S: tf.function # Create the model, optimizer and metrics inside the `tf.distribute.Strategy` # scope, so that the variables can be mirrored on each device. with strategy.scope(): model = create_model() optimizer = tf.keras.optimizers.Adam() training_loss = tf.keras.metrics.Mean('training_loss', dtype=tf.float32) training_accuracy = tf.keras.metrics.SparseCategoricalAccuracy( 'training_accuracy', dtype=tf.float32) # Calculate per replica batch size, and distribute the `tf.data.Dataset`s # on each TPU worker. per_replica_batch_size = batch_size // strategy.num_replicas_in_sync train_dataset = strategy.experimental_distribute_datasets_from_function( lambda _: get_dataset(per_replica_batch_size, is_training=True)) @tf.function def train_step(iterator): """The step function for one training step.""" def step_fn(inputs): """The computation to run on each TPU device.""" images, labels = inputs with tf.GradientTape() as tape: logits = model(images, training=True) loss = tf.keras.losses.sparse_categorical_crossentropy( labels, logits, from_logits=True) loss = tf.nn.compute_average_loss(loss, global_batch_size=batch_size) grads = tape.gradient(loss, model.trainable_variables) optimizer.apply_gradients(list(zip(grads, model.trainable_variables))) training_loss.update_state(loss * strategy.num_replicas_in_sync) training_accuracy.update_state(labels, logits) strategy.run(step_fn, args=(next(iterator),)) WARNING:tensorflow:From /tmpfs/tmp/ipykernel_9094/1509474074.py:14: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version. Instructions for updating: rename to distribute_datasets_from_function WARNING:tensorflow:From /tmpfs/tmp/ipykernel_9094/1509474074.py:14: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version. Instructions for updating: rename to distribute_datasets_from_function Emva koko, uqhagamshelane ne-training loop: steps_per_eval = 10000 // batch_size train_iterator = iter(train_dataset) for epoch in range(5): print('Epoch: {}/5'.format(epoch)) for step in range(steps_per_epoch): train_step(train_iterator) print('Current step: {}, training loss: {}, accuracy: {}%'.format( optimizer.iterations.numpy(), round(float(training_loss.result()), 4), round(float(training_accuracy.result()) * 100, 2))) training_loss.reset_states() training_accuracy.reset_states() Epoch: 0/5 Current step: 300, training loss: 0.1465, accuracy: 95.4% Epoch: 1/5 Current step: 600, training loss: 0.035, accuracy: 98.94% Epoch: 2/5 Current step: 900, training loss: 0.0197, accuracy: 99.39% Epoch: 3/5 Current step: 1200, training loss: 0.0126, accuracy: 99.59% Epoch: 4/5 Current step: 1500, training loss: 0.0109, accuracy: 99.64% Ukuphucula ukusebenza ngeengxaki ezininzi ngaphakathi tf.function Uyakwazi ukwandisa ukusebenza ngokuchofoza iminyango ezininzi ngaphakathi Ukusetyenziswa kwimveliso Qhagamshelana nathi Ngaphandle , kwaye i-AutoGraph ithetha ukuba a on the TPU worker. Ungafunda ngaphezulu malunga S kwi umnikelo tf.function Strategy.run tf.range tf.function tf.while_loop tf.function Ukusebenza Kuhle nge tf.function Nangona ukusebenza olungcono, kukho iintlawulo kunye ne-methode ehambelanayo kunxhomekeke ekugqibeleni iinyathelo elinye ngaphakathi Ukusebenza iinyathelo ezininzi kwi-a I-Python ye-Python ye-Python ye-Python ye-Python ye-Python ye-Python ye-Python ye-Python tf.function tf.function @tf.function def train_multiple_steps(iterator, steps): """The step function for one training step.""" def step_fn(inputs): """The computation to run on each TPU device.""" images, labels = inputs with tf.GradientTape() as tape: logits = model(images, training=True) loss = tf.keras.losses.sparse_categorical_crossentropy( labels, logits, from_logits=True) loss = tf.nn.compute_average_loss(loss, global_batch_size=batch_size) grads = tape.gradient(loss, model.trainable_variables) optimizer.apply_gradients(list(zip(grads, model.trainable_variables))) training_loss.update_state(loss * strategy.num_replicas_in_sync) training_accuracy.update_state(labels, logits) for _ in tf.range(steps): strategy.run(step_fn, args=(next(iterator),)) # Convert `steps_per_epoch` to `tf.Tensor` so the `tf.function` won't get # retraced if the value changes. train_multiple_steps(train_iterator, tf.convert_to_tensor(steps_per_epoch)) print('Current step: {}, training loss: {}, accuracy: {}%'.format( optimizer.iterations.numpy(), round(float(training_loss.result()), 4), round(float(training_accuracy.result()) * 100, 2))) Current step: 1800, training loss: 0.009, accuracy: 99.72% Ukucaciswa Ukufundisa ngakumbi malunga ne-Cloud TPUs kunye nenkqubo yokusebenzisa: I-Google Cloud TPU: Ikhaya ye-Google Cloud TPU. : Google Cloud TPU documentation, which includes: Google Cloud TPU documentation : An overview of working with Cloud TPUs. Introduction to Cloud TPU : Quickstart introductions to working with Cloud TPU VMs using TensorFlow and other main machine learning frameworks. Cloud TPU quickstarts I-Google Cloud TPU Colab Notebooks: Imibuzo yemfundo yeend-to-end. I-Google Cloud TPU Performance Guide: Ukuphucula ukusebenza kwe-Cloud TPU ngokugqithisa i-Cloud TPU configuration parameters ye-application yakho Ukuqeqeshwa kwe-Distributed kunye ne-TensorFlow: Indlela yokusebenzisa i-strategies ye-distribution - kuquka i-tf.distribute.TPUStrategy - kunye neengxelo zibonisa iimveliso ezilungileyo. Ukufakelwa kwe-TPU: I-TensorFlow ibandakanya inkxaso ye-training embeddings kwi-TPU nge-tf.tpu.experimental.embedding. Ukongezelela, i-TensorFlow Recommenders ibandakanya i-tfrs.layers.embedding.TPUEmbedding. I-embeddings ibandakanya i-representations ezincinane kunye ne-density, i-capture of similarities and relationships between features. I-TPU-specific embedding support ye-TensorFlow ibandakanya i-embeddings ezininzi kunokuba i-memory ye-TPU enye, kwaye usebenzisa i-inputs ezincinane kunye ne-ragged kwi-TPUs. I-TPU Research Cloud (TRC): I-TRC ibonelela abaculi ukuba bafumane ukufikelela kwi-cluster ye-1,000 i-cloud TPU devices. Okokuqala ifumaneka kwi-website yeTensorFlow, le nqaku ifumaneka apha phantsi kwegama elitsha kwaye ifumaneka phantsi kweCC BY 4.0. Iimveliso ze-code zithunyelwe phantsi kwe-Apache 2.0 License. Okokuqala ifumaneka kwi-website yeTensorFlow, le nqaku ifumaneka apha phantsi kwegama elitsha kwaye ifumaneka phantsi kweCC BY 4.0. Iimveliso ze-code zithunyelwe phantsi kwe-Apache 2.0 License. TensorFlow TensorFlow

Ukwandisa isantya seTensorFlow kwi-50% nge-TPU Tricks

About Author

IZIMVO

ZIJONGE IIMPAWU

ELI NQAKU LINIKEZELWE KU

Related Stories

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps