Ongeza kasi ya mafunzo ya TensorFlow kwa 50% na Vidokezo hivi vya TPU

Maelezo ya Maudhui Muundo wa Kuanzishwa kwa TPU » Ujenzi wa vifaa vya manual Mkakati wa Usambazaji Ufafanuzi wa TPUs Ufafanuzi wa Mfano wa Keras Kuendesha Dataset Kufundisha mfano kwa kutumia API za Kiwango cha juu cha Keras Kuendesha mfano kwa kutumia kozi ya mafunzo Kuboresha utendaji kwa hatua nyingi ndani ya tf.function Hatua ya baadaye Mwongozo huu unaonyesha jinsi ya kufanya mafunzo ya msingi juu ya na TPU Pods, mkusanyiko wa vifaa vya TPU vinavyounganishwa na viungo vya mtandao vya kasi, na na mazoezi ya mazoezi. Njia ya Usindikaji wa Tensor (TPUs) tf.keras TPUs ni mzunguko wa kuunganishwa wa programu maalum (ASICs) uliotengenezwa na Google, unaotumika kuharakisha kazi za kujifunza mashine. kwa ajili ya na ya . Maelezo ya Google Colab Utafiti wa Cloud ya TPU Cloud ya TPU Muundo wa Kabla ya kuendesha notebook hii ya Colab, hakikisha kuwa accelerator yako ya vifaa ni TPU kwa kuangalia mipangilio ya notebook yako: > ya > ya > ya . Runtime Change runtime type Hardware accelerator TPU Import baadhi ya maktaba muhimu, ikiwa ni pamoja na Datasets TensorFlow: import tensorflow as tf import os import tensorflow_datasets as tfds 2023-06-09 12:13:32.486552: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT Utaratibu wa TPU TPUs ni ya kawaida wafanyakazi, ambao ni tofauti na mchakato wa ndani unaendesha programu ya Python ya mtumiaji. Kwa hiyo, unahitaji kufanya kazi ya awali ya kuunganisha kwenye cluster ya mbali na kuanza TPUs. Mjadala wa ni anwani maalum tu kwa Colab. Ikiwa unaendesha msimbo wako kwenye Google Compute Engine (GCE), unapaswa badala yake kutuma jina la TPU yako ya Cloud. Cloud ya TPU tpu tf.distribute.cluster_resolver.TPUClusterResolver Kumbuka: Msimbo wa ufumbuzi wa TPU lazima uwe mwanzo wa mpango wako. Kumbuka: Msimbo wa ufumbuzi wa TPU lazima uwe mwanzo wa mpango wako. resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='') tf.config.experimental_connect_to_cluster(resolver) # This is the TPU initialization code that has to be at the beginning. tf.tpu.experimental.initialize_tpu_system(resolver) print("All devices: ", tf.config.list_logical_devices('TPU')) INFO:tensorflow:Deallocate tpu buffers before initializing tpu system. 2023-06-09 12:13:34.011755: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected INFO:tensorflow:Deallocate tpu buffers before initializing tpu system. INFO:tensorflow:Initializing the TPU system: grpc://10.25.167.66:8470 INFO:tensorflow:Initializing the TPU system: grpc://10.25.167.66:8470 INFO:tensorflow:Finished initializing TPU system. INFO:tensorflow:Finished initializing TPU system. All devices: [LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:0', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:1', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:2', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:3', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:4', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:5', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:6', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:7', device_type='TPU')] Ujenzi wa vifaa vya manual Baada ya TPU imeanzishwa, unaweza kutumia nafasi ya kifaa kwa mkono ili kuweka hesabu kwenye kifaa kimoja cha TPU: a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]]) with tf.device('/TPU:0'): c = tf.matmul(a, b) print("c device: ", c.device) print(c) c device: /job:worker/replica:0/task:0/device:TPU:0 tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32) Mkakati wa Usambazaji Kwa kawaida, unaweza kuendesha mfano wako kwenye TPU kadhaa kwa njia ya data parallel. Ili kueneza mfano wako kwenye TPU kadhaa (kama vile GPU kadhaa au mashine kadhaa), TensorFlow inatoa API. Unaweza kubadilisha mkakati wako wa usambazaji na mfano utaendeshwa kwenye kifaa chochote kinachotolewa (TPU). mwongozo wa. tf.distribute.Strategy Mafunzo ya kusambazwa na TensorFlow Kutumia kwa TPUs kutoa utekelezaji wao wenyewe wa ufanisi wote-kuvunja na shughuli nyingine za pamoja juu ya viungo vingi vya TPU, ambazo hutumiwa katika . tf.distribute.TPUStrategy TPUStrategy Ili kuonyesha hili, kuunda a kwa ajili ya: tf.distribute.TPUStrategy strategy = tf.distribute.TPUStrategy(resolver) INFO:tensorflow:Found TPU system: INFO:tensorflow:Found TPU system: INFO:tensorflow:*** Num TPU Cores: 8 INFO:tensorflow:*** Num TPU Cores: 8 INFO:tensorflow:*** Num TPU Workers: 1 INFO:tensorflow:*** Num TPU Workers: 1 INFO:tensorflow:*** Num TPU Cores Per Worker: 8 INFO:tensorflow:*** Num TPU Cores Per Worker: 8 INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0) Ili kurekebisha hesabu ili inaweza kutumika katika viungo vyote vya TPU, unaweza kuhamisha kwenye API. Chini ni mfano ambao unaonyesha vichwa vyote vilivyopokea inapatikana sawa na kutekeleza multiplication ya matrix juu ya kila kiini huru. Matoleo yatakuwa thamani kutoka kwa replicas zote. Strategy.run (a, b) @tf.function def matmul_fn(x, y): z = tf.matmul(x, y) return z z = strategy.run(matmul_fn, args=(a, b)) print(z) PerReplica:{ 0: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 1: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 2: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 3: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 4: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 5: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 6: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 7: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32) } Ufafanuzi wa TPUs Baada ya kufunika dhana za msingi, fikiria mfano maalum zaidi. sehemu hii inaonyesha jinsi ya kutumia mkakati wa usambazaji— —kufundisha mfano wa Keras kwenye Cloud TPU. tf.distribute.TPUStrategy Ufafanuzi wa Mfano wa Keras Kuanza na ufafanuzi wa a Mfano wa Keras kwa utambulisho wa picha kwenye dataset ya MNIST. Haina tofauti na kile unachotumia kama ulikuwa unajifunza juu ya CPU au GPU. Kumbuka kwamba uumbaji wa mfano wa Keras unahitaji kuwa ndani ya , hivyo mabadiliko yanaweza kuundwa kwenye kila kifaa cha TPU. sehemu zingine za msimbo hazihitaji kuwa ndani ya ya scope. Sequential Strategy.scope Strategy def create_model(): return tf.keras.Sequential( [tf.keras.layers.Conv2D(256, 3, activation='relu', input_shape=(28, 28, 1)), tf.keras.layers.Conv2D(256, 3, activation='relu'), tf.keras.layers.Flatten(), tf.keras.layers.Dense(256, activation='relu'), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(10)]) Kuendesha Dataset Matumizi ya ufanisi wa API ni muhimu wakati wa kutumia Cloud TPU. Unaweza kujifunza zaidi kuhusu utendaji wa dataset katika . tf.data.Dataset Mwongozo wa Ufanisi wa Input Pipeline Kama wewe ni kutumia , unahitaji kuhifadhi faili zote za data zilizosoma na TensorFlow katika Kama wewe ni kutumia Unaweza kuunda kazi na uendelee haraka na matumizi ya masanduku. atakuwa na uwezo wa kuchunguza na kurekebisha wakati kila siku, hali ya hali ya hewa na risasi kamera magari yao. ya nyaraka. Node ya TPU Dataset Uhifadhi wa wingu wa Google (GCS) wa wa wa wa wa Mfumo wa TPU Architecture Kwa kesi nyingi za matumizi, inashauriwa kubadilisha data yako katika Kuweka na kutumia A Hebu tuone jinsi vile nyongeza za kazi na jinsi salama kwa kompyuta. ☆☆ kwa maelezo juu ya jinsi ya kufanya hivyo. Huu sio mahitaji ngumu na unaweza kutumia wasomaji wengine wa dataset, kama vile au . TFRecord tf.data.TFRecordDataset TFRecord na tf.Mfano wa mafunzo tf.data.FixedLengthRecordDataset tf.data.TextLineDataset Unaweza kupakia mikutano yote ya data ndogo kwenye kumbukumbu kwa kutumia . tf.data.Dataset.cache Bila kujali aina ya data iliyotumika, inashauriwa sana kutumia faili kubwa katika utaratibu wa 100MB. Hii ni muhimu hasa katika mazingira haya ya mtandao, kwa sababu juu ya kufungua faili ni kubwa sana. Kama ilivyoonyeshwa katika msimbo hapa chini, unapaswa kutumia Datasets ya Tensorflow Moduli ya kupata nakala ya mafunzo ya MNIST na data ya mtihani. imeelezwa kutumia nakala ambayo inapatikana katika bucket ya GCS ya umma. Ikiwa huna ufafanuzi huu, TPU haitakuwa na uwezo wa kufikia data iliyotolewa. tfds.load try_gcs def get_dataset(batch_size, is_training=True): split = 'train' if is_training else 'test' dataset, info = tfds.load(name='mnist', split=split, with_info=True, as_supervised=True, try_gcs=True) # Normalize the input data. def scale(image, label): image = tf.cast(image, tf.float32) image /= 255.0 return image, label dataset = dataset.map(scale) # Only shuffle and repeat the dataset in training. The advantage of having an # infinite dataset for training is to avoid the potential last partial batch # in each epoch, so that you don't need to think about scaling the gradients # based on the actual batch size. if is_training: dataset = dataset.shuffle(10000) dataset = dataset.repeat() dataset = dataset.batch(batch_size) return dataset Kufundisha mfano kwa kutumia API za Kiwango cha juu cha Keras Unaweza kuunda kazi na uendelee haraka na ya Hakuna kitu cha TPU maalum katika hatua hii—unaandika msimbo kama unatumia GPU kadhaa na Badala ya Unaweza kujifunza zaidi katika Maelezo ya Tutorial. Model.fit Model.compile MirroredStrategy TPUStrategy Mafunzo kwa ajili ya Keras with strategy.scope(): model = create_model() model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['sparse_categorical_accuracy']) batch_size = 200 steps_per_epoch = 60000 // batch_size validation_steps = 10000 // batch_size train_dataset = get_dataset(batch_size, is_training=True) test_dataset = get_dataset(batch_size, is_training=False) model.fit(train_dataset, epochs=5, steps_per_epoch=steps_per_epoch, validation_data=test_dataset, validation_steps=validation_steps) Epoch 1/5 300/300 [==============================] - 17s 32ms/step - loss: 0.1235 - sparse_categorical_accuracy: 0.9620 - val_loss: 0.0462 - val_sparse_categorical_accuracy: 0.9856 Epoch 2/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0333 - sparse_categorical_accuracy: 0.9894 - val_loss: 0.0401 - val_sparse_categorical_accuracy: 0.9878 Epoch 3/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0186 - sparse_categorical_accuracy: 0.9938 - val_loss: 0.0352 - val_sparse_categorical_accuracy: 0.9900 Epoch 4/5 300/300 [==============================] - 7s 25ms/step - loss: 0.0127 - sparse_categorical_accuracy: 0.9957 - val_loss: 0.0482 - val_sparse_categorical_accuracy: 0.9879 Epoch 5/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0111 - sparse_categorical_accuracy: 0.9962 - val_loss: 0.0448 - val_sparse_categorical_accuracy: 0.9894 Ili kupunguza juu ya Python na kuongeza utendaji wa TPU yako, kupita katika Maneno ya Keras Katika mfano huu, inaongeza upatikanaji wa takriban 50%: steps_per_execution Model.compile with strategy.scope(): model = create_model() model.compile(optimizer='adam', # Anything between 2 and `steps_per_epoch` could help here. steps_per_execution = 50, loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['sparse_categorical_accuracy']) model.fit(train_dataset, epochs=5, steps_per_epoch=steps_per_epoch, validation_data=test_dataset, validation_steps=validation_steps) Epoch 1/5 300/300 [==============================] - 14s 45ms/step - loss: 0.1306 - sparse_categorical_accuracy: 0.9591 - val_loss: 0.0420 - val_sparse_categorical_accuracy: 0.9863 Epoch 2/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0333 - sparse_categorical_accuracy: 0.9900 - val_loss: 0.0502 - val_sparse_categorical_accuracy: 0.9846 Epoch 3/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0193 - sparse_categorical_accuracy: 0.9936 - val_loss: 0.0406 - val_sparse_categorical_accuracy: 0.9879 Epoch 4/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0135 - sparse_categorical_accuracy: 0.9955 - val_loss: 0.0416 - val_sparse_categorical_accuracy: 0.9882 Epoch 5/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0110 - sparse_categorical_accuracy: 0.9962 - val_loss: 0.0463 - val_sparse_categorical_accuracy: 0.9882 Kuendesha mfano kwa kutumia kozi ya mafunzo Unaweza pia kuunda na kufundisha mfano wako kwa kutumia na ya Kuanzisha mlinzi juu ya msalaba wa msalaba. at API kwa ajili ya usambazaji Kumbuka kwamba katika mfano wa chini, ukubwa wa mfululizo uliingia katika ni ukubwa wa mfululizo wa replica badala ya ukubwa wa mfululizo wa kimataifa. Kujifunza zaidi, angalia Maelezo ya Tutorial. tf.function tf.distribute Strategy.experimental_distribute_datasets_from_function tf.data.Dataset Dataset Mafunzo ya kawaida na tf.distribute.Strategy Kwanza, unahitaji kubadilisha muundo, data na ya S: tf.function # Create the model, optimizer and metrics inside the `tf.distribute.Strategy` # scope, so that the variables can be mirrored on each device. with strategy.scope(): model = create_model() optimizer = tf.keras.optimizers.Adam() training_loss = tf.keras.metrics.Mean('training_loss', dtype=tf.float32) training_accuracy = tf.keras.metrics.SparseCategoricalAccuracy( 'training_accuracy', dtype=tf.float32) # Calculate per replica batch size, and distribute the `tf.data.Dataset`s # on each TPU worker. per_replica_batch_size = batch_size // strategy.num_replicas_in_sync train_dataset = strategy.experimental_distribute_datasets_from_function( lambda _: get_dataset(per_replica_batch_size, is_training=True)) @tf.function def train_step(iterator): """The step function for one training step.""" def step_fn(inputs): """The computation to run on each TPU device.""" images, labels = inputs with tf.GradientTape() as tape: logits = model(images, training=True) loss = tf.keras.losses.sparse_categorical_crossentropy( labels, logits, from_logits=True) loss = tf.nn.compute_average_loss(loss, global_batch_size=batch_size) grads = tape.gradient(loss, model.trainable_variables) optimizer.apply_gradients(list(zip(grads, model.trainable_variables))) training_loss.update_state(loss * strategy.num_replicas_in_sync) training_accuracy.update_state(labels, logits) strategy.run(step_fn, args=(next(iterator),)) WARNING:tensorflow:From /tmpfs/tmp/ipykernel_9094/1509474074.py:14: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version. Instructions for updating: rename to distribute_datasets_from_function WARNING:tensorflow:From /tmpfs/tmp/ipykernel_9094/1509474074.py:14: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version. Instructions for updating: rename to distribute_datasets_from_function Kisha, uendesha mzunguko wa mafunzo: steps_per_eval = 10000 // batch_size train_iterator = iter(train_dataset) for epoch in range(5): print('Epoch: {}/5'.format(epoch)) for step in range(steps_per_epoch): train_step(train_iterator) print('Current step: {}, training loss: {}, accuracy: {}%'.format( optimizer.iterations.numpy(), round(float(training_loss.result()), 4), round(float(training_accuracy.result()) * 100, 2))) training_loss.reset_states() training_accuracy.reset_states() Epoch: 0/5 Current step: 300, training loss: 0.1465, accuracy: 95.4% Epoch: 1/5 Current step: 600, training loss: 0.035, accuracy: 98.94% Epoch: 2/5 Current step: 900, training loss: 0.0197, accuracy: 99.39% Epoch: 3/5 Current step: 1200, training loss: 0.0126, accuracy: 99.59% Epoch: 4/5 Current step: 1500, training loss: 0.0109, accuracy: 99.64% Kuboresha utendaji kwa hatua nyingi ndani tf.function Unaweza kuboresha utendaji kwa kuendesha hatua nyingi ndani ya Hii inafanywa kwa kuunganisha Piga simu kwa a Ndani ya , na AutoGraph itabadilisha kwa a juu ya wafanyakazi wa TPU. Unaweza kujifunza zaidi juu ya S katika mwongozo wa. tf.function Strategy.run tf.range tf.function tf.while_loop tf.function Ufanisi bora zaidi kwa tf.function Licha ya utendaji bora, kuna ushindani na mbinu hii ikilinganishwa na kukimbia hatua moja ndani ya Kuendesha hatua nyingi katika a ni kidogo - huwezi kuendesha mambo kwa hamu au msimbo wa Python ndani ya hatua. tf.function tf.function @tf.function def train_multiple_steps(iterator, steps): """The step function for one training step.""" def step_fn(inputs): """The computation to run on each TPU device.""" images, labels = inputs with tf.GradientTape() as tape: logits = model(images, training=True) loss = tf.keras.losses.sparse_categorical_crossentropy( labels, logits, from_logits=True) loss = tf.nn.compute_average_loss(loss, global_batch_size=batch_size) grads = tape.gradient(loss, model.trainable_variables) optimizer.apply_gradients(list(zip(grads, model.trainable_variables))) training_loss.update_state(loss * strategy.num_replicas_in_sync) training_accuracy.update_state(labels, logits) for _ in tf.range(steps): strategy.run(step_fn, args=(next(iterator),)) # Convert `steps_per_epoch` to `tf.Tensor` so the `tf.function` won't get # retraced if the value changes. train_multiple_steps(train_iterator, tf.convert_to_tensor(steps_per_epoch)) print('Current step: {}, training loss: {}, accuracy: {}%'.format( optimizer.iterations.numpy(), round(float(training_loss.result()), 4), round(float(training_accuracy.result()) * 100, 2))) Current step: 1800, training loss: 0.009, accuracy: 99.72% Hatua ya baadaye Jifunze zaidi kuhusu Cloud TPUs na jinsi ya kutumia: Google Cloud TPU: Tovuti ya Google Cloud TPU. : Google Cloud TPU documentation, which includes: Google Cloud TPU documentation : An overview of working with Cloud TPUs. Introduction to Cloud TPU : Quickstart introductions to working with Cloud TPU VMs using TensorFlow and other main machine learning frameworks. Cloud TPU quickstarts Google Cloud TPU Colab Notebooks: Mifano ya mafunzo ya mwisho. Mwongozo wa utendaji wa Google Cloud TPU: Kuboresha utendaji wa Cloud TPU kwa kurekebisha vigezo vya usanidi wa Cloud TPU kwa programu yako Mafunzo ya kusambazwa na TensorFlow: Jinsi ya kutumia mikakati ya usambazaji - ikiwa ni pamoja na tf.distribute.TPUStrategy - na mifano ya mazoea mazuri. TPU embeddings: TensorFlow inajumuisha msaada maalum kwa ajili ya mafunzo embeddings juu ya TPUs kupitia tf.tpu.experimental.embedding. Zaidi ya hayo, TensorFlow Recommenders ina tfrs.layers.embedding.TPUEmbedding. Embeddings hutoa uwakilishi wa ufanisi na wa jumla, kukamata utofauti na uhusiano kati ya vipengele. Msaada wa TPU-specific embeddings wa TensorFlow inakuwezesha kufundisha embeddings ambayo ni kubwa kuliko kumbukumbu ya kifaa cha TPU moja, na kutumia ufumbuzi mdogo na wa rangi kwenye TPUs. TPU Utafiti Cloud (TRC): TRC inaruhusu watafiti kuomba upatikanaji wa cluster ya vifaa vya Cloud TPU zaidi ya 1,000. Awali iliyochapishwa kwenye tovuti ya TensorFlow, makala hii inaonekana hapa chini ya kichwa kipya na inatolewa kwa leseni chini ya CC BY 4.0. Awali iliyochapishwa kwenye tovuti ya TensorFlow, makala hii inaonekana hapa chini ya kichwa kipya na inatolewa kwa leseni chini ya CC BY 4.0. Uwezo wa TensorFlow Uwezo wa TensorFlow