Tensor Slicing και Εισαγωγή Δεδομένων Εύκολη με TensorFlow

Περιεχόμενο Επισκόπηση Εισαγωγή ΕΞΑΓΩΓΗ ΤΕΝΣΟΡ Εισαγωγή δεδομένων σε τέντες Περαιτέρω ανάγνωση και πόροι Όταν εργάζεστε σε εφαρμογές ML, όπως η ανίχνευση αντικειμένων και η NLP, είναι μερικές φορές απαραίτητο να εργαστείτε με υποτμήματα (τμήματα) τενζορών. Για παράδειγμα, αν η αρχιτεκτονική του μοντέλου σας περιλαμβάνει δρομολόγηση, όπου ένα στρώμα μπορεί να ελέγχει ποιο παράδειγμα εκπαίδευσης δρομολογείται στο επόμενο στρώμα. Για παράδειγμα, μπορείτε να δημιουργήσετε δεδομένα κατάρτισης από μια λίστα προτάσεων επιλέγοντας έναν δείκτη λέξης για να καλύψετε σε κάθε πρόταση, παίρνοντας τη λέξη ως ετικέτα και στη συνέχεια αντικαθιστώντας την επιλεγμένη λέξη με ένα token μάσκας. Σε αυτόν τον οδηγό, θα μάθετε πώς να χρησιμοποιείτε τα API του TensorFlow για: Εξαγωγή τεμαχίων από έναν τεντωτή Εισαγωγή δεδομένων σε συγκεκριμένους δείκτες σε έναν τεντωτή Αυτός ο οδηγός προϋποθέτει εξοικείωση με την ένδειξη των τενόντων. Διαβάστε τις ενότητες και Οδηγίες πριν ξεκινήσετε με αυτόν τον οδηγό. Τενέρ TensorFlow αριθμοί εγκατάσταση import tensorflow as tf import numpy as np 2024-08-15 02:59:22.387959: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-08-15 02:59:22.409014: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-08-15 02:59:22.415391: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered ΕΞΑΓΩΓΗ ΤΕΝΣΟΡ Εκτελέστε το Tensor Slicing χρησιμοποιώντας . tf.slice t1 = tf.constant([0, 1, 2, 3, 4, 5, 6, 7]) print(tf.slice(t1, begin=[1], size=[3])) tf.Tensor([1 2 3], shape=(3,), dtype=int32) WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1723690765.029742 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.033507 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.037154 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.040869 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.052765 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.056359 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.059719 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.063138 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.066532 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.070040 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.073650 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690765.077235 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.300773 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.302797 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.304766 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.306851 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.308843 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.310723 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.312589 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.314574 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.316423 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.318291 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.320168 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.322150 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.360968 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.362946 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.364849 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.366867 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.368710 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.370591 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.372447 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.374435 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.376302 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.378677 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.380966 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723690766.383365 169903 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 Εναλλακτικά, μπορείτε να χρησιμοποιήσετε μια πιο πυθωνική σύνταξη. Σημειώστε ότι οι φέτες τένζερ είναι ομοιόμορφα διατεταγμένες σε μια περιοχή start-stop. print(t1[1:4]) tf.Tensor([1 2 3], shape=(3,), dtype=int32) print(t1[-3:]) tf.Tensor([5 6 7], shape=(3,), dtype=int32) Για δύο διαστάσεις, μπορείτε να χρησιμοποιήσετε κάτι σαν: t2 = tf.constant([[0, 1, 2, 3, 4], [5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19]]) print(t2[:-1, 1:3]) tf.Tensor( [[ 1 2] [ 6 7] [11 12]], shape=(3, 2), dtype=int32) Μπορείτε να χρησιμοποιήσετε Ακόμη και σε υψηλότερες διαστάσεις. tf.slice t3 = tf.constant([[[1, 3, 5, 7], [9, 11, 13, 15]], [[17, 19, 21, 23], [25, 27, 29, 31]] ]) print(tf.slice(t3, begin=[1, 1, 0], size=[1, 1, 2])) tf.Tensor([[[25 27]]], shape=(1, 1, 2), dtype=int32) Μπορείτε επίσης να χρησιμοποιήσετε για να εξαγάγετε φέτες των τενζορών "πατώντας" πάνω από τις διαστάσεις των τενζορών. tf.strided_slice Χρησιμοποιήστε για να εξαγάγετε συγκεκριμένους δείκτες από έναν μόνο άξονα ενός τεντωτή. tf.gather print(tf.gather(t1, indices=[0, 3, 6])) # This is similar to doing t1[::3] tf.Tensor([0 3 6], shape=(3,), dtype=int32) Δεν απαιτείται η ομοιόμορφη κατανομή των δεικτών. tf.gather alphabet = tf.constant(list('abcdefghijklmnopqrstuvwxyz')) print(tf.gather(alphabet, indices=[2, 0, 19, 18])) tf.Tensor([b'c' b'a' b't' b's'], shape=(4,), dtype=string) Για να εξαγάγετε φέτες από πολλαπλούς άξονες ενός τενζορ, χρησιμοποιήστε Αυτό είναι χρήσιμο όταν θέλετε να συγκεντρώσετε τα στοιχεία ενός πίνακα αντί μόνο για τις γραμμές ή τις στήλες του. tf.gather_nd t4 = tf.constant([[0, 5], [1, 6], [2, 7], [3, 8], [4, 9]]) print(tf.gather_nd(t4, indices=[[2], [3], [0]])) tf.Tensor( [[2 7] [3 8] [0 5]], shape=(3, 2), dtype=int32) t5 = np.reshape(np.arange(18), [2, 3, 3]) print(tf.gather_nd(t5, indices=[[0, 0, 0], [1, 2, 1]])) tf.Tensor([ 0 16], shape=(2,), dtype=int64) # Return a list of two matrices print(tf.gather_nd(t5, indices=[[[0, 0], [0, 2]], [[1, 0], [1, 2]]])) tf.Tensor( [[[ 0 1 2] [ 6 7 8]] [[ 9 10 11] [15 16 17]]], shape=(2, 2, 3), dtype=int64) # Return one matrix print(tf.gather_nd(t5, indices=[[0, 0], [0, 2], [1, 0], [1, 2]])) tf.Tensor( [[ 0 1 2] [ 6 7 8] [ 9 10 11] [15 16 17]], shape=(4, 3), dtype=int64) Εισαγωγή δεδομένων σε τέντες Χρησιμοποιήστε για να εισαγάγετε δεδομένα σε συγκεκριμένες φέτες / δείκτες ενός τενζορ. Σημειώστε ότι ο τενζορ στον οποίο εισάγετε τιμές είναι μηδενικός. tf.scatter_nd t6 = tf.constant([10]) indices = tf.constant([[1], [3], [5], [7], [9]]) data = tf.constant([2, 4, 6, 8, 10]) print(tf.scatter_nd(indices=indices, updates=data, shape=t6)) tf.Tensor([ 0 2 0 4 0 6 0 8 0 10], shape=(10,), dtype=int32) Μέθοδοι όπως που απαιτούν μηδενικές αρχικές τάσεις είναι παρόμοιες με τις σπάνιες αρχικές τάσεις. και να μιμούνται τη συμπεριφορά των sparse tensor ops. tf.scatter_nd tf.gather_nd tf.scatter_nd Εξετάστε ένα παράδειγμα όπου χτίζετε έναν σπάνιο τενζολάρο χρησιμοποιώντας αυτές τις δύο μεθόδους σε συνδυασμό. # Gather values from one tensor by specifying indices new_indices = tf.constant([[0, 2], [2, 1], [3, 3]]) t7 = tf.gather_nd(t2, indices=new_indices) # Add these values into a new tensor t8 = tf.scatter_nd(indices=new_indices, updates=t7, shape=tf.constant([4, 5])) print(t8) tf.Tensor( [[ 0 0 2 0 0] [ 0 0 0 0 0] [ 0 11 0 0 0] [ 0 0 0 18 0]], shape=(4, 5), dtype=int32) Αυτό είναι παρόμοιο με: t9 = tf.SparseTensor(indices=[[0, 2], [2, 1], [3, 3]], values=[2, 11, 18], dense_shape=[4, 5]) print(t9) SparseTensor(indices=tf.Tensor( [[0 2] [2 1] [3 3]], shape=(3, 2), dtype=int64), values=tf.Tensor([ 2 11 18], shape=(3,), dtype=int32), dense_shape=tf.Tensor([4 5], shape=(2,), dtype=int64)) # Convert the sparse tensor into a dense tensor t10 = tf.sparse.to_dense(t9) print(t10) tf.Tensor( [[ 0 0 2 0 0] [ 0 0 0 0 0] [ 0 11 0 0 0] [ 0 0 0 18 0]], shape=(4, 5), dtype=int32) Για να εισαγάγετε δεδομένα σε έναν τέντορα με προκαθορισμένες τιμές, χρησιμοποιήστε . tf.tensor_scatter_nd_add t11 = tf.constant([[2, 7, 0], [9, 0, 1], [0, 3, 8]]) # Convert the tensor into a magic square by inserting numbers at appropriate indices t12 = tf.tensor_scatter_nd_add(t11, indices=[[0, 2], [1, 1], [2, 0]], updates=[6, 5, 4]) print(t12) tf.Tensor( [[2 7 6] [9 5 1] [4 3 8]], shape=(3, 3), dtype=int32) Ομοίως, η χρήση για να αφαιρέσετε τιμές από έναν τέντορα με προκαθορισμένες τιμές. tf.tensor_scatter_nd_sub # Convert the tensor into an identity matrix t13 = tf.tensor_scatter_nd_sub(t11, indices=[[0, 0], [0, 1], [1, 0], [1, 1], [1, 2], [2, 1], [2, 2]], updates=[1, 7, 9, -1, 1, 3, 7]) print(t13) tf.Tensor( [[1 0 0] [0 1 0] [0 0 1]], shape=(3, 3), dtype=int32) Χρησιμοποιήστε να αντιγράψετε ελάχιστες τιμές στοιχείων από έναν τενζολάρο σε άλλον. tf.tensor_scatter_nd_min t14 = tf.constant([[-2, -7, 0], [-9, 0, 1], [0, -3, -8]]) t15 = tf.tensor_scatter_nd_min(t14, indices=[[0, 2], [1, 1], [2, 0]], updates=[-6, -5, -4]) print(t15) tf.Tensor( [[-2 -7 -6] [-9 -5 1] [-4 -3 -8]], shape=(3, 3), dtype=int32) Ομοίως, η χρήση για να αντιγράψετε τις μέγιστες τιμές του στοιχείου από έναν τενόρα σε άλλον. tf.tensor_scatter_nd_max t16 = tf.tensor_scatter_nd_max(t14, indices=[[0, 2], [1, 1], [2, 0]], updates=[6, 5, 4]) print(t16) tf.Tensor( [[-2 -7 6] [-9 5 1] [ 4 -3 -8]], shape=(3, 3), dtype=int32) Περαιτέρω ανάγνωση και πόροι Σε αυτό το εγχειρίδιο, μάθατε πώς να χρησιμοποιείτε τις επιλογές κοπής τένζερ που είναι διαθέσιμες με το TensorFlow για να ασκήσετε λεπτότερο έλεγχο επί των στοιχείων στους τένζερ σας. Ελέγξτε τις επιλογές κοπής που είναι διαθέσιμες με το TensorFlow NumPy, όπως tf.experimental.numpy.take_along_axis και tf.experimental.numpy.take. Επίσης, δείτε τον οδηγό Tensor και τον οδηγό Μεταβλητών. Αρχικά δημοσιευμένο στην ιστοσελίδα TensorFlow, αυτό το άρθρο εμφανίζεται εδώ κάτω από μια νέα επικεφαλίδα και είναι αδειοδοτημένο υπό την άδεια CC BY 4.0. Αρχικά δημοσιευμένο στην ιστοσελίδα TensorFlow, αυτό το άρθρο εμφανίζεται εδώ κάτω από μια νέα επικεφαλίδα και είναι αδειοδοτημένο υπό την άδεια CC BY 4.0. TensorFlow TensorFlow