Boost TensorFlow fòmasyon vitès pa 50% ak sa yo TPU Trucs

Kontni vèsyon Pwodwi Initialization nan TPU' Manuèl aparèy plasman Strateji Distribisyon Klasifikasyon sou TPUs Defini yon modèl Keras Loading nan Dataset Train modèl la lè l sèvi avèk API segondè nivo nan Keras Tren modèl la lè l sèvi avèk yon lyen fòmasyon Custom amelyore pèfòmans ak plizyè etap nan tf.function Prèske etap Guid sa a montre ki jan yo fè fòmasyon prensipal sou ak TPU Pods, yon koleksyon de TPU aparèy konekte pa dedye segondè-vitès rezo enteryè, ak Epitou fòmasyon koutim. Tensor Processing Units (TPUs) se yon tf.keras TPU yo se konpitè entegre aplikasyon-espesifik (ASICs) ki te devlope pa Custom nan Google ki itilize pou akselere workloads machin aprantisaj. Yo disponib nan nan E . Google kolab TPU Rechèch Cloud Cloud nan TPU Pwodwi Anvan ou kouri sa a Colab notebook, asire w ke akseleratè a aparèy ou se yon TPU pa tcheke anviwònman an nan notebook ou: > nan > nan > nan . Runtime Change runtime type Hardware accelerator TPU Importe kèk livrezon ki nesesè, ki gen ladan Datasets TensorFlow: import tensorflow as tf import os import tensorflow_datasets as tfds 2023-06-09 12:13:32.486552: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT TPU inisyasyon TPU yo se tipik travayè, ki diferan de pwosesis lokal la kouri pwogram Python nan itilizatè a. Se poutèt sa, ou bezwen fè kèk initialization travay yo konekte nan cluster a distan ak initialize TPUs yo. Remake ke Argòm nan se yon adrès espesyal sèlman pou Colab. Si ou kouri kòd ou sou Google Compute Engine (GCE), ou ta dwe pase nan non an nan Cloud TPU ou. Cloud nan TPU tpu tf.distribute.cluster_resolver.TPUClusterResolver Remak: Kòd la TPU initialization dwe nan kòmansman an nan pwogram ou. Kòd Inicializasyon TPU a dwe nan kòmansman an nan pwogram ou. Note: resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='') tf.config.experimental_connect_to_cluster(resolver) # This is the TPU initialization code that has to be at the beginning. tf.tpu.experimental.initialize_tpu_system(resolver) print("All devices: ", tf.config.list_logical_devices('TPU')) INFO:tensorflow:Deallocate tpu buffers before initializing tpu system. 2023-06-09 12:13:34.011755: E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:266] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected INFO:tensorflow:Deallocate tpu buffers before initializing tpu system. INFO:tensorflow:Initializing the TPU system: grpc://10.25.167.66:8470 INFO:tensorflow:Initializing the TPU system: grpc://10.25.167.66:8470 INFO:tensorflow:Finished initializing TPU system. INFO:tensorflow:Finished initializing TPU system. All devices: [LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:0', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:1', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:2', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:3', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:4', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:5', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:6', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:7', device_type='TPU')] Manuèl aparèy plasman Yon fwa TPU a se inisyalize, ou ka sèvi ak plasman aparèy manyen mete kalkil la sou yon sèl aparèy TPU: a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]) b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]]) with tf.device('/TPU:0'): c = tf.matmul(a, b) print("c device: ", c.device) print(c) c device: /job:worker/replica:0/task:0/device:TPU:0 tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32) Strateji Distribisyon Souvan, ou kouri modèl ou sou plizyè TPUs nan yon fason done paralèl. Pou distribye modèl ou sou plizyè TPUs (tou menm jan ak plizyè GPUs oswa plizyè machin), TensorFlow ofri API. Ou ka ranplase estrateji distribisyon ou ak modèl la pral kouri sou nenpòt ki aparèy (TPU). Aprann plis nan Gide nan. tf.distribute.Strategy Distribye fòmasyon ak TensorFlow Sèvi ak opsyon implemente fòmasyon distribiye sinchrone. TPU yo bay pwòp aplikasyon yo nan efikas tout-rediksyon ak lòt operasyon kolektiv atravè plizyè kouti TPU, ki yo itilize nan . tf.distribute.TPUStrategy TPUStrategy Pou demontre sa a, kreye yon Objè nan: tf.distribute.TPUStrategy strategy = tf.distribute.TPUStrategy(resolver) INFO:tensorflow:Found TPU system: INFO:tensorflow:Found TPU system: INFO:tensorflow:*** Num TPU Cores: 8 INFO:tensorflow:*** Num TPU Cores: 8 INFO:tensorflow:*** Num TPU Workers: 1 INFO:tensorflow:*** Num TPU Workers: 1 INFO:tensorflow:*** Num TPU Cores Per Worker: 8 INFO:tensorflow:*** Num TPU Cores Per Worker: 8 INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:localhost/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:CPU:0, CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:0, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:1, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:2, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:3, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:4, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:5, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:6, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU:7, TPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:TPU_SYSTEM:0, TPU_SYSTEM, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0) INFO:tensorflow:*** Available Device: _DeviceAttributes(/job:worker/replica:0/task:0/device:XLA_CPU:0, XLA_CPU, 0, 0) Pou replike yon kalkil pou li ka kouri nan tout koutim TPU, ou ka pase li nan API. Ki anba a se yon egzanp ki montre tout koòdone receiving menm entwodiksyon ak egzekite multiplikasyon matris sou chak koòdone otomatikman. Rezilta yo pral valè yo soti nan tout replika yo. Strategy.run (a, b) @tf.function def matmul_fn(x, y): z = tf.matmul(x, y) return z z = strategy.run(matmul_fn, args=(a, b)) print(z) PerReplica:{ 0: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 1: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 2: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 3: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 4: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 5: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 6: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32), 7: tf.Tensor( [[22. 28.] [49. 64.]], shape=(2, 2), dtype=float32) } Klasifikasyon sou TPUs Apre kouvri konpozisyon yo debaz, konsidere yon egzanp plis kontan. Sa a seksyon montre ki jan yo sèvi ak estrateji distribisyon- - tren yon modèl Keras sou yon Cloud TPU. tf.distribute.TPUStrategy Defini yon modèl Keras Kòmanse ak yon definisyon nan a modèl Keras pou klasifikasyon imaj sou dataset la MNIST. Li se pa diferan de sa ou ta itilize si ou te fòmasyon sou CPUs oswa GPUs. Remake ke kreyasyon modèl Keras ta dwe nan , se konsa varyab yo ka kreye sou chak aparèy TPU. Yon lòt pati nan kòd la pa nesesè yo dwe nan Scope nan. Sequential Strategy.scope Strategy def create_model(): return tf.keras.Sequential( [tf.keras.layers.Conv2D(256, 3, activation='relu', input_shape=(28, 28, 1)), tf.keras.layers.Conv2D(256, 3, activation='relu'), tf.keras.layers.Flatten(), tf.keras.layers.Dense(256, activation='relu'), tf.keras.layers.Dense(128, activation='relu'), tf.keras.layers.Dense(10)]) Loading nan Dataset Efikas itilize nan API se kritik lè l sèvi avèk yon TPU Cloud. Ou ka aprann plis sou pèfòmans dataset nan . tf.data.Dataset Input pipeline pèfòmans gid Si ou ap itilize , ou bezwen sove tout dosye done li pa TensorFlow nan Si ou ap itilize , ou ka sove done kote ou vle. Pou plis enfòmasyon sou TPU Nodes ak TPU VMs, tanpri referans Dokiman nan. Nòd TPU Dataset Google Cloud Storage (GCS) nan konpitè VMs nan TPU TPU sistèm Architecture Pou pifò ka itilize, li rekòmande pou konvèti done ou nan fòme ak itilize a pou yo li. Tcheke nan pou detay sou ki jan yo fè sa. Li pa yon kondisyon difisil epi ou ka sèvi ak lòt lektè dataset, tankou ou . TFRecord tf.data.TFRecordDataset TFRecord ak tf.Example tutoryèl tf.data.FixedLengthRecordDataset tf.data.TextLineDataset Èske ou ka chaje tout ti seri done nan memwa lè l sèvi avèk . tf.data.Dataset.cache Nenpòt fòma done ki te itilize, li se trè rekòmande ke ou sèvi ak dosye gwo nan lòd nan 100MB. Sa a se espesyalman enpòtan nan anviwònman an rezo, paske overhead la nan louvri yon dosye se anpil pi wo. Kòm montre nan kòd la anba a, ou ta dwe sèvi ak Datasets Tensorflow modil pou jwenn yon kopi nan MNIST fòmasyon ak done tès. Remake ke se espesifye yo sèvi ak yon kopi ki disponib nan yon bwat piblik GCS. Si ou pa espesifye sa a, TPU a pa pral kapab aksè nan done yo download. tfds.load try_gcs def get_dataset(batch_size, is_training=True): split = 'train' if is_training else 'test' dataset, info = tfds.load(name='mnist', split=split, with_info=True, as_supervised=True, try_gcs=True) # Normalize the input data. def scale(image, label): image = tf.cast(image, tf.float32) image /= 255.0 return image, label dataset = dataset.map(scale) # Only shuffle and repeat the dataset in training. The advantage of having an # infinite dataset for training is to avoid the potential last partial batch # in each epoch, so that you don't need to think about scaling the gradients # based on the actual batch size. if is_training: dataset = dataset.shuffle(10000) dataset = dataset.repeat() dataset = dataset.batch(batch_size) return dataset Train modèl la lè l sèvi avèk API segondè nivo nan Keras Èske ou ka fòme modèl ou ak Keras ak APIs. Pa gen anyen TPU-spesifik nan etap sa a - ou ekri kòd la tankou si w ap itilize plizyè GPU ak yon Nan plas la nan Ou ka aprann plis nan Tutorial nan. Model.fit Model.compile MirroredStrategy TPUStrategy Distribye fòmasyon ak Keras with strategy.scope(): model = create_model() model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['sparse_categorical_accuracy']) batch_size = 200 steps_per_epoch = 60000 // batch_size validation_steps = 10000 // batch_size train_dataset = get_dataset(batch_size, is_training=True) test_dataset = get_dataset(batch_size, is_training=False) model.fit(train_dataset, epochs=5, steps_per_epoch=steps_per_epoch, validation_data=test_dataset, validation_steps=validation_steps) Epoch 1/5 300/300 [==============================] - 17s 32ms/step - loss: 0.1235 - sparse_categorical_accuracy: 0.9620 - val_loss: 0.0462 - val_sparse_categorical_accuracy: 0.9856 Epoch 2/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0333 - sparse_categorical_accuracy: 0.9894 - val_loss: 0.0401 - val_sparse_categorical_accuracy: 0.9878 Epoch 3/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0186 - sparse_categorical_accuracy: 0.9938 - val_loss: 0.0352 - val_sparse_categorical_accuracy: 0.9900 Epoch 4/5 300/300 [==============================] - 7s 25ms/step - loss: 0.0127 - sparse_categorical_accuracy: 0.9957 - val_loss: 0.0482 - val_sparse_categorical_accuracy: 0.9879 Epoch 5/5 300/300 [==============================] - 7s 24ms/step - loss: 0.0111 - sparse_categorical_accuracy: 0.9962 - val_loss: 0.0448 - val_sparse_categorical_accuracy: 0.9894 Pou redwi overhead nan Python ak maksimize pèfòmans nan TPU ou a, pase nan Arguments pou nan Nan egzanp sa a, li ogmante pèfòmans nan apeprè 50%: steps_per_execution Model.compile with strategy.scope(): model = create_model() model.compile(optimizer='adam', # Anything between 2 and `steps_per_epoch` could help here. steps_per_execution = 50, loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['sparse_categorical_accuracy']) model.fit(train_dataset, epochs=5, steps_per_epoch=steps_per_epoch, validation_data=test_dataset, validation_steps=validation_steps) Epoch 1/5 300/300 [==============================] - 14s 45ms/step - loss: 0.1306 - sparse_categorical_accuracy: 0.9591 - val_loss: 0.0420 - val_sparse_categorical_accuracy: 0.9863 Epoch 2/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0333 - sparse_categorical_accuracy: 0.9900 - val_loss: 0.0502 - val_sparse_categorical_accuracy: 0.9846 Epoch 3/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0193 - sparse_categorical_accuracy: 0.9936 - val_loss: 0.0406 - val_sparse_categorical_accuracy: 0.9879 Epoch 4/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0135 - sparse_categorical_accuracy: 0.9955 - val_loss: 0.0416 - val_sparse_categorical_accuracy: 0.9882 Epoch 5/5 300/300 [==============================] - 3s 10ms/step - loss: 0.0110 - sparse_categorical_accuracy: 0.9962 - val_loss: 0.0463 - val_sparse_categorical_accuracy: 0.9882 Tren modèl la lè l sèvi avèk yon lyen fòmasyon Custom Ou ka tou kreye ak fòmasyon modèl ou lè l sèvi avèk ak APIs dirèkteman. Ou ka sèvi ak API pou distribye Dapre yon fonksyon dataset. Remake ke nan egzanp la anba a, gwosè a batch la te pase nan se gwosè a batch pou chak replike anvan gwosè a batch mondyal la. Pou aprann plis, tcheke nan Tutorial nan. tf.function tf.distribute Strategy.experimental_distribute_datasets_from_function tf.data.Dataset Dataset Custom fòmasyon ak tf.distribute.Strategy Premye, kreye modèl la, dataset yo ak s nan: tf.function # Create the model, optimizer and metrics inside the `tf.distribute.Strategy` # scope, so that the variables can be mirrored on each device. with strategy.scope(): model = create_model() optimizer = tf.keras.optimizers.Adam() training_loss = tf.keras.metrics.Mean('training_loss', dtype=tf.float32) training_accuracy = tf.keras.metrics.SparseCategoricalAccuracy( 'training_accuracy', dtype=tf.float32) # Calculate per replica batch size, and distribute the `tf.data.Dataset`s # on each TPU worker. per_replica_batch_size = batch_size // strategy.num_replicas_in_sync train_dataset = strategy.experimental_distribute_datasets_from_function( lambda _: get_dataset(per_replica_batch_size, is_training=True)) @tf.function def train_step(iterator): """The step function for one training step.""" def step_fn(inputs): """The computation to run on each TPU device.""" images, labels = inputs with tf.GradientTape() as tape: logits = model(images, training=True) loss = tf.keras.losses.sparse_categorical_crossentropy( labels, logits, from_logits=True) loss = tf.nn.compute_average_loss(loss, global_batch_size=batch_size) grads = tape.gradient(loss, model.trainable_variables) optimizer.apply_gradients(list(zip(grads, model.trainable_variables))) training_loss.update_state(loss * strategy.num_replicas_in_sync) training_accuracy.update_state(labels, logits) strategy.run(step_fn, args=(next(iterator),)) WARNING:tensorflow:From /tmpfs/tmp/ipykernel_9094/1509474074.py:14: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version. Instructions for updating: rename to distribute_datasets_from_function WARNING:tensorflow:From /tmpfs/tmp/ipykernel_9094/1509474074.py:14: StrategyBase.experimental_distribute_datasets_from_function (from tensorflow.python.distribute.distribute_lib) is deprecated and will be removed in a future version. Instructions for updating: rename to distribute_datasets_from_function Lè sa a, kouri kou fòmasyon: steps_per_eval = 10000 // batch_size train_iterator = iter(train_dataset) for epoch in range(5): print('Epoch: {}/5'.format(epoch)) for step in range(steps_per_epoch): train_step(train_iterator) print('Current step: {}, training loss: {}, accuracy: {}%'.format( optimizer.iterations.numpy(), round(float(training_loss.result()), 4), round(float(training_accuracy.result()) * 100, 2))) training_loss.reset_states() training_accuracy.reset_states() Epoch: 0/5 Current step: 300, training loss: 0.1465, accuracy: 95.4% Epoch: 1/5 Current step: 600, training loss: 0.035, accuracy: 98.94% Epoch: 2/5 Current step: 900, training loss: 0.0197, accuracy: 99.39% Epoch: 3/5 Current step: 1200, training loss: 0.0126, accuracy: 99.59% Epoch: 4/5 Current step: 1500, training loss: 0.0109, accuracy: 99.64% amelyore pèfòmans ak plizyè etap nan tf.function Èske ou ka amelyore pèfòmans pa kouri plizyè etap nan yon Sa a se reyalize pa ranplase Telefòn ak yon Enfòmasyon , ak AutoGraph pral konvèti li nan yon sou travayè TPU. Ou ka aprann plis sou S nan Gide nan. tf.function Strategy.run tf.range tf.function tf.while_loop tf.function Pi bon pèfòmans ak tf.function Malgre nan pèfòmans amelyore, gen kompromis ak metòd sa a konpare ak kouri yon sèl etap nan yon Kòmanse plizyè etap nan yon se mwens fleksib - ou pa ka kouri bagay yo enteresan oswa arbitre kòd Python nan etap yo. tf.function tf.function @tf.function def train_multiple_steps(iterator, steps): """The step function for one training step.""" def step_fn(inputs): """The computation to run on each TPU device.""" images, labels = inputs with tf.GradientTape() as tape: logits = model(images, training=True) loss = tf.keras.losses.sparse_categorical_crossentropy( labels, logits, from_logits=True) loss = tf.nn.compute_average_loss(loss, global_batch_size=batch_size) grads = tape.gradient(loss, model.trainable_variables) optimizer.apply_gradients(list(zip(grads, model.trainable_variables))) training_loss.update_state(loss * strategy.num_replicas_in_sync) training_accuracy.update_state(labels, logits) for _ in tf.range(steps): strategy.run(step_fn, args=(next(iterator),)) # Convert `steps_per_epoch` to `tf.Tensor` so the `tf.function` won't get # retraced if the value changes. train_multiple_steps(train_iterator, tf.convert_to_tensor(steps_per_epoch)) print('Current step: {}, training loss: {}, accuracy: {}%'.format( optimizer.iterations.numpy(), round(float(training_loss.result()), 4), round(float(training_accuracy.result()) * 100, 2))) Current step: 1800, training loss: 0.009, accuracy: 99.72% Prèske etap Pou aprann plis sou Cloud TPUs ak ki jan yo sèvi ak yo: Google Cloud TPU: Google Cloud TPU se yon sit entènèt. : Google Cloud TPU documentation, which includes: Google Cloud TPU documentation : An overview of working with Cloud TPUs. Introduction to Cloud TPU : Quickstart introductions to working with Cloud TPU VMs using TensorFlow and other main machine learning frameworks. Cloud TPU quickstarts Google Cloud TPU Colab notebooks: End-to-end egzanp fòmasyon. Google Cloud TPU pèfòmans gid: amelyore pèfòmans nan Cloud TPU plis pa ajiste paramèt konfigirasyon nan Cloud TPU pou aplikasyon ou Distribisyon fòmasyon ak TensorFlow: Ki jan yo sèvi ak estrateji distribisyon - ki gen ladan tf.distribute.TPUStrategy - ak egzanp ki montre pratik yo pi byen. TPU embeddings: TensorFlow gen ladan sipò espesyalize pou fòmasyon embeddings sou TPUs via tf.tpu.experimental.embedding. Anplis de sa, TensorFlow Recommenders gen tfrs.layers.embedding.TPUEmbedding. Embeddings bay reprezantasyon efikas ak dense, capturing konplisite ak relasyon ant karakteristik yo. sipò a TPU-spesifik embeddings nan TensorFlow pèmèt ou fòmasyon embeddings ki pi gwo pase memwa a nan yon sèl aparèy TPU, ak pou sèvi ak enpòsyon sparse ak ragged sou TPUs. TPU Rechèch Cloud (TRC): TRC pèmèt rechèchè aplike pou aksè nan yon klavye nan plis pase 1,000 Cloud TPU aparèy. Orijinèlman pibliye sou sit entènèt la TensorFlow, atik sa a ap parèt isit la anba yon tit nouvo ak se lisans anba CC BY 4.0. Samples kòd pataje anba lisans la Apache 2.0. Orijinèlman pibliye sou sit entènèt la TensorFlow, atik sa a ap parèt isit la anba yon tit nouvo ak se lisans anba CC BY 4.0. Samples kòd pataje anba lisans la Apache 2.0. Pwodwi pou Pwodwi pou