Slučajnost definirana najpopularnijim jezikom strojnog učenja

Sadržaj Pregled postavljanje Tf.random.Generator razred Stvaranje neovisnih struja slučajnih brojeva Interakcija s tf.function Interakcija s distribucijskim strategijama Ušteda generatora Neovisni RNG Algoritmi generala XLA uređaji TensorFlow pruža skup pseudo-slučajnih brojeva generatora (RNG), u Ovaj dokument opisuje kako možete kontrolirati generatore slučajnih brojeva i kako ti generatori međusobno djeluju s drugim podsustavima tenzorskog toka. tf.random Napomena: Ne jamči se da će slučajni brojevi biti dosljedni u različitim verzijama TensorFlowa. Slučajni brojevi nisu zajamčeni da će biti dosljedni u svim verzijama TensorFlow-a. Note: Version Compatibility Kompatibilnost verzija TensorFlow pruža dva pristupa za kontrolu procesa generiranja slučajnih brojeva: Svaki takav objekt održava stanje (u tf.Variable) koje će se promijeniti nakon svake generacije brojeva. Pozivanje tih funkcija s istim argumentima (koji uključuju sjeme) i na istom uređaju uvijek će proizvesti iste rezultate. Upozorenje: Stari RNG-ovi iz TF 1.x kao što su tf.random.uniform i tf.random.normal još nisu deprecirani, ali su snažno obeshrabreni. Stari RNGs iz TF 1.x kao što su i Još uvijek nisu oslabljeni, ali su oštro obeshrabreni. Warning: tf.random.uniform tf.random.normal postavljanje import tensorflow as tf # Creates some virtual devices (cpu:0, cpu:1, etc.) for using distribution strategy physical_devices = tf.config.list_physical_devices("CPU") tf.config.experimental.set_virtual_device_configuration( physical_devices[0], [ tf.config.experimental.VirtualDeviceConfiguration(), tf.config.experimental.VirtualDeviceConfiguration(), tf.config.experimental.VirtualDeviceConfiguration() ]) 2024-08-15 01:43:41.157432: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered 2024-08-15 01:43:41.178819: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered 2024-08-15 01:43:41.185039: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered WARNING: All log messages before absl::InitializeLog() is called are written to STDERR I0000 00:00:1723686223.758551 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.762466 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.765643 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.769228 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.780976 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.784469 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.787369 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.790779 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.794250 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.797851 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.800627 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686223.804177 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 Naš Klasa tf.random.Generator Tf.random.Generator uređaji Naš Klasa se koristi u slučajevima kada želite da svaki RNG poziv proizvodi različite rezultate. Objekt) koji će se ažurirati svaki put kada se generiraju slučajni brojevi. uživa u svim uslugama koje pruža kao što su jednostavna kontrolna točka, automatsko upravljanje ovisnošću i sigurnost žica. tf.random.Generator tf.Variable tf.Variable tf.Variable Možete dobiti a ručno kreiranje objekta klase ili poziva Da biste dobili podrazumijevani globalni generator: tf.random.Generator tf.random.get_global_generator() g1 = tf.random.Generator.from_seed(1) print(g1.normal(shape=[2, 3])) g2 = tf.random.get_global_generator() print(g2.normal(shape=[2, 3])) I0000 00:00:1723686225.040869 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.043016 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.045041 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.047115 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.049184 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.051188 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.053162 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.055160 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.057105 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.059113 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.061029 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 tf.Tensor( [[ 0.43842277 -0.53439844 -0.07710262] [ 1.5658045 -0.1012345 -0.2744976 ]], shape=(2, 3), dtype=float32) tf.Tensor( [[ 1.3061213 0.6299361 0.52625704] [ 1.3733886 0.29277426 -0.7945693 ]], shape=(2, 3), dtype=float32) I0000 00:00:1723686225.063243 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.101904 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.103956 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.105919 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.107957 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.109922 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.111909 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.113815 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.115823 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.117783 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.120279 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.122678 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 I0000 00:00:1723686225.125117 55391 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355 Postoji nekoliko načina za stvaranje objekta generatora. najjednostavniji je , kao što je prikazano gore, to stvara generator iz sjemena. Također postoji opcijski argument koji je RNG algoritam koji će koristiti ovaj generator: Generator.from_seed from_seed alg g1 = tf.random.Generator.from_seed(1, alg='philox') print(g1.normal(shape=[2, 3])) tf.Tensor( [[ 0.43842277 -0.53439844 -0.07710262] [ 1.5658045 -0.1012345 -0.2744976 ]], shape=(2, 3), dtype=float32) Vidi se Odjeljak ispod za više informacija o tome. Algoritmi Drugi način za stvaranje generatora je s Generator stvoren na ovaj način započinje iz nedeterminističkog stanja, ovisno o npr. vremenu i OS-u. Generator.from_non_deterministic_state g = tf.random.Generator.from_non_deterministic_state() print(g.normal(shape=[2, 3])) tf.Tensor( [[ 0.9104948 -0.23143363 -0.09841432] [-0.91448975 0.1579936 1.3923475 ]], shape=(2, 3), dtype=float32) Postoje i drugi načini za stvaranje generatora, na primjer iz eksplicitnih stanja, koji nisu obuhvaćeni ovim vodičem. Kada se koristi da biste dobili globalni generator, morate biti oprezni o postavljanju uređaja. globalni generator je stvoren (iz ne-determinističkog stanja) prvi put poziva i stavlja se na podrazumijevani uređaj na taj poziv. tako, na primjer, ako je prva stranica koju pozovete To je unutar a Općenito, globalni generator će biti smješten na GPU, a pomoću globalnog generatora kasnije iz CPU-a dobit će se GPU-to-CPU kopija. tf.random.get_global_generator tf.random.get_global_generator tf.random.get_global_generator tf.device("gpu") Postoji i funkcija za zamjenu globalnog generatora s drugim objektom generatora. Ova funkcija treba koristiti s oprezom, iako, jer stari globalni generator može biti zarobljen od strane (kao slaba referenca), a zamjena će uzrokovati da se sakupi smeće, razbijanje Bolji način ponovnog postavljanja globalnog generatora je korištenje jedne od funkcija "reset" kao što su: , koji neće stvoriti nove objekte generatora. tf.random.set_global_generator tf.function tf.function Generator.reset_from_seed g = tf.random.Generator.from_seed(1) print(g.normal([])) print(g.normal([])) g.reset_from_seed(1) print(g.normal([])) tf.Tensor(0.43842277, shape=(), dtype=float32) tf.Tensor(1.6272374, shape=(), dtype=float32) tf.Tensor(0.43842277, shape=(), dtype=float32) Stvaranje neovisnih struja slučajnih brojeva U mnogim aplikacijama potrebno je nekoliko neovisnih tokova slučajnih brojeva, neovisnih u smislu da se neće preklapati i neće imati statistički otkrivene korelacije. stvaranje višestrukih generatora koji su zajamčeni kao neovisni jedni od drugih (tj. stvaranje neovisnih tokova). Generator.split g = tf.random.Generator.from_seed(1) print(g.normal([])) new_gs = g.split(3) for new_g in new_gs: print(new_g.normal([])) print(g.normal([])) tf.Tensor(0.43842277, shape=(), dtype=float32) tf.Tensor(2.536413, shape=(), dtype=float32) tf.Tensor(0.33186463, shape=(), dtype=float32) tf.Tensor(-0.07144657, shape=(), dtype=float32) tf.Tensor(-0.79253083, shape=(), dtype=float32) će se promijeniti stanje generatora na kojem se zove ( u gore navedenom primjeru), sličan RNG metodi kao što je Osim što su neovisni jedni o drugima, novi generatori ( ) također su zajamčeni da budu neovisni od starog ( ) i split g normal new_gs g Spajanje novih generatora također je korisno kada želite osigurati da je generator koji koristite na istom uređaju kao i drugi izračuni, kako bi se izbjeglo prekomjerno kopiranje uređaja. with tf.device("cpu"): # change "cpu" to the device you want g = tf.random.get_global_generator().split(1)[0] print(g.normal([])) # use of g won't cause cross-device copy, unlike the global generator tf.Tensor(-0.66787744, shape=(), dtype=float32) Napomena: Teoretski možete koristiti konstruktore kao što je from_seed umjesto podjele ovdje kako biste dobili novi generator, ali time gubite jamstvo da je novi generator neovisan od globalnog generatora. U teoriji, možete koristiti konstruktore kao što su Umjesto Ovdje ćete dobiti novi generator, ali time ćete izgubiti jamstvo da je novi generator neovisan o globalnom generatoru. Note: from_seed split Možete podijeliti recursivno, pozivajući Na split generatorima ne postoje granice (ne ograničavajući preopterećenje čitavog broja) na dubini recursija. split Interakcija s tf.function Slijedite ista pravila kao i Kada se koristi s To uključuje tri aspekta. tf.random.Generator tf.Variable tf.function Izgradnja izvana generatora tf.function tf. funkcija Možete koristiti generator stvoren izvan njega. tf.function g = tf.random.Generator.from_seed(1) @tf.function def foo(): return g.normal([]) print(foo()) tf.Tensor(0.43842277, shape=(), dtype=float32) Korisnik mora osigurati da je objekt generatora još uvijek živ (ne sakuplja otpad) kada se pozove funkcija. Izgradnja generatora unutar tf. funkcija tf. funkcija Stvaranje generatora unutar a To se može dogoditi samo tijekom prvog rada funkcije. tf.function g = None @tf.function def foo(): global g if g is None: g = tf.random.Generator.from_seed(1) return g.normal([]) print(foo()) print(foo()) tf.Tensor(0.43842277, shape=(), dtype=float32) tf.Tensor(1.6272374, shape=(), dtype=float32) Generatori kao argumenti za tf. funkcija tf. funkcija Kada se koristi kao argument za , različiti objekti generatora će uzrokovati povlačenje . tf.function tf.function num_traces = 0 @tf.function def foo(g): global num_traces num_traces += 1 return g.normal([]) foo(tf.random.Generator.from_seed(1)) foo(tf.random.Generator.from_seed(2)) print(num_traces) 2 Imajte na umu da je takvo ponašanje u skladu s : tf.Variable num_traces = 0 @tf.function def foo(v): global num_traces num_traces += 1 return v.read_value() foo(tf.Variable(1)) foo(tf.Variable(2)) print(num_traces) 1 Interakcija s distribucijskim strategijama Postoje dva načina na koje Razgovarajte o distribucijskim strategijama. Generator Stvaranje generatora izvan distribucijskih strategija Ako se generator stvara izvan opsega strategije, pristup svih replika generatoru bit će serijaliziran, pa će replike dobiti različite slučajne brojeve. g = tf.random.Generator.from_seed(1) strat = tf.distribute.MirroredStrategy(devices=["cpu:0", "cpu:1"]) with strat.scope(): def f(): print(g.normal([])) results = strat.run(f) INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `run` inside a tf.function to get the best performance. tf.Tensor(0.43842274, shape=(), dtype=float32) tf.Tensor(1.6272374, shape=(), dtype=float32) Imajte na umu da ova upotreba može imati problema s učinkom jer se uređaj generatora razlikuje od replika. Stvaranje generatora unutar distribucijskih strategija Ako je generator stvoren unutar opsega strategije, svaka replika će dobiti drugačiji i neovisni tok slučajnih brojeva. strat = tf.distribute.MirroredStrategy(devices=["cpu:0", "cpu:1"]) with strat.scope(): g = tf.random.Generator.from_seed(1) print(strat.run(lambda: g.normal([]))) print(strat.run(lambda: g.normal([]))) INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `run` inside a tf.function to get the best performance. PerReplica:{ 0: tf.Tensor(-0.87930447, shape=(), dtype=float32), 1: tf.Tensor(0.020661574, shape=(), dtype=float32) } WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `run` inside a tf.function to get the best performance. PerReplica:{ 0: tf.Tensor(-1.5822568, shape=(), dtype=float32), 1: tf.Tensor(0.77539235, shape=(), dtype=float32) } Napomena: Trenutno tf.random.Generator ne pruža mogućnost da različite replike dobiju identične (umjesto različitih) tokove (što tehnički nije teško). Trenutno ne pruža mogućnost da različite replike dobiju identične (umjesto različitih) tokove (što tehnički nije teško). Note: tf.random.Generator Ako je generator sjeme (npr. stvoren od strane ), slučajni brojevi određuju sjeme, iako različite replike dobivaju različite i nekorelirane brojeve. Može se misliti o slučajnom broju generiranom na repliki kao hašu ID-a replike i "primarnom" slučajnom broju koji je zajednički svim replicima. Generator.from_seed Također se može stvoriti unutar : tf.random.Generator Strategy.run strat = tf.distribute.MirroredStrategy(devices=["cpu:0", "cpu:1"]) with strat.scope(): def f(): g = tf.random.Generator.from_seed(1) a = g.normal([]) b = g.normal([]) return tf.stack([a, b]) print(strat.run(f)) print(strat.run(f)) INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `run` inside a tf.function to get the best performance. PerReplica:{ 0: tf.Tensor([-0.87930447 -1.5822568 ], shape=(2,), dtype=float32), 1: tf.Tensor([0.02066157 0.77539235], shape=(2,), dtype=float32) } WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `run` inside a tf.function to get the best performance. PerReplica:{ 0: tf.Tensor([-0.87930447 -1.5822568 ], shape=(2,), dtype=float32), 1: tf.Tensor([0.02066157 0.77539235], shape=(2,), dtype=float32) } Više ne preporučujemo prolaz Kao argumenti za Zato što Općenito se očekuje da argumenti budu tenzori, a ne generatori. tf.random.Generator Strategy.run Strategy.run Ušteda generatora Općenito, za štednju ili serializiranje možete se nositi s Na isti način postupali bi i s ili a U TF-u postoje dva mehanizma serializiranja: i . tf.random.Generator tf.Variable tf.Module Checkpoint Spasitelj Checkpoint Generatori se mogu slobodno čuvati i vraćati pomoću Stream slučajnih brojeva iz točke vraćanja bit će isti kao i od točke spašavanja. tf.train.Checkpoint filename = "./checkpoint" g = tf.random.Generator.from_seed(1) cp = tf.train.Checkpoint(generator=g) print(g.normal([])) tf.Tensor(0.43842277, shape=(), dtype=float32) cp.write(filename) print("RNG stream from saving point:") print(g.normal([])) print(g.normal([])) RNG stream from saving point: tf.Tensor(1.6272374, shape=(), dtype=float32) tf.Tensor(1.6307176, shape=(), dtype=float32) cp.restore(filename) print("RNG stream from restoring point:") print(g.normal([])) print(g.normal([])) RNG stream from restoring point: tf.Tensor(1.6272374, shape=(), dtype=float32) tf.Tensor(1.6307176, shape=(), dtype=float32) Također možete sačuvati i vratiti unutar strategije distribucije: filename = "./checkpoint" strat = tf.distribute.MirroredStrategy(devices=["cpu:0", "cpu:1"]) with strat.scope(): g = tf.random.Generator.from_seed(1) cp = tf.train.Checkpoint(my_generator=g) print(strat.run(lambda: g.normal([]))) INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') PerReplica:{ 0: tf.Tensor(-0.87930447, shape=(), dtype=float32), 1: tf.Tensor(0.020661574, shape=(), dtype=float32) } with strat.scope(): cp.write(filename) print("RNG stream from saving point:") print(strat.run(lambda: g.normal([]))) print(strat.run(lambda: g.normal([]))) RNG stream from saving point: PerReplica:{ 0: tf.Tensor(-1.5822568, shape=(), dtype=float32), 1: tf.Tensor(0.77539235, shape=(), dtype=float32) } PerReplica:{ 0: tf.Tensor(-0.5039703, shape=(), dtype=float32), 1: tf.Tensor(0.1251838, shape=(), dtype=float32) } with strat.scope(): cp.restore(filename) print("RNG stream from restoring point:") print(strat.run(lambda: g.normal([]))) print(strat.run(lambda: g.normal([]))) RNG stream from restoring point: PerReplica:{ 0: tf.Tensor(-1.5822568, shape=(), dtype=float32), 1: tf.Tensor(0.77539235, shape=(), dtype=float32) } PerReplica:{ 0: tf.Tensor(-0.5039703, shape=(), dtype=float32), 1: tf.Tensor(0.1251838, shape=(), dtype=float32) } Trebali biste se pobrinuti da se replike ne razlikuju u povijesti RNG poziva (npr. jedna replika obavlja jedan RNG poziv, dok druga obavlja dva RNG poziva) prije čuvanja. (koji samo čuva stanje prve replike) neće ispravno vratiti sve replike. tf.train.Checkpoint Također možete vratiti sačuvanu kontrolnu točku na drugu strategiju distribucije s različitim brojem replika. objekat stvoren u strategiji može se koristiti samo u istoj strategiji, kako bi se vratio na drugu strategiju, morate stvoriti novu u cilju ostvarivanja ciljeva i nove Za to, kao što je prikazano u ovom primjeru: tf.random.Generator tf.random.Generator tf.train.Checkpoint filename = "./checkpoint" strat1 = tf.distribute.MirroredStrategy(devices=["cpu:0", "cpu:1"]) with strat1.scope(): g1 = tf.random.Generator.from_seed(1) cp1 = tf.train.Checkpoint(my_generator=g1) print(strat1.run(lambda: g1.normal([]))) INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') PerReplica:{ 0: tf.Tensor(-0.87930447, shape=(), dtype=float32), 1: tf.Tensor(0.020661574, shape=(), dtype=float32) } with strat1.scope(): cp1.write(filename) print("RNG stream from saving point:") print(strat1.run(lambda: g1.normal([]))) print(strat1.run(lambda: g1.normal([]))) RNG stream from saving point: PerReplica:{ 0: tf.Tensor(-1.5822568, shape=(), dtype=float32), 1: tf.Tensor(0.77539235, shape=(), dtype=float32) } PerReplica:{ 0: tf.Tensor(-0.5039703, shape=(), dtype=float32), 1: tf.Tensor(0.1251838, shape=(), dtype=float32) } strat2 = tf.distribute.MirroredStrategy(devices=["cpu:0", "cpu:1", "cpu:2"]) with strat2.scope(): g2 = tf.random.Generator.from_seed(1) cp2 = tf.train.Checkpoint(my_generator=g2) cp2.restore(filename) print("RNG stream from restoring point:") print(strat2.run(lambda: g2.normal([]))) print(strat2.run(lambda: g2.normal([]))) INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1', '/job:localhost/replica:0/task:0/device:CPU:2') INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1', '/job:localhost/replica:0/task:0/device:CPU:2') RNG stream from restoring point: PerReplica:{ 0: tf.Tensor(-1.5822568, shape=(), dtype=float32), 1: tf.Tensor(0.77539235, shape=(), dtype=float32), 2: tf.Tensor(0.6851049, shape=(), dtype=float32) } PerReplica:{ 0: tf.Tensor(-0.5039703, shape=(), dtype=float32), 1: tf.Tensor(0.1251838, shape=(), dtype=float32), 2: tf.Tensor(-0.58519536, shape=(), dtype=float32) } Iako i Postoje različiti predmeti od i , povezani su putem zajedničke kontrolne točke datoteke ime objekta Preklapanje replika između strategija (npr. i Ovo jamstvo ne pokriva slučaj kada se generator čuva u okviru strategije i vraća izvan bilo kojeg okvira strategije ili obrnuto, jer se uređaj izvan strategija tretira kao drugačiji od bilo koje replike u strategiji. g1 cp1 g2 cp2 filename my_generator cpu:0 cpu:1 Spasitelj generator se može stvoriti unutar opsega strategije. štednja se također može dogoditi unutar opsega strategije. tf.random.Generator filename = "./saved_model" class MyModule(tf.Module): def __init__(self): super(MyModule, self).__init__() self.g = tf.random.Generator.from_seed(0) @tf.function def __call__(self): return self.g.normal([]) @tf.function def state(self): return self.g.state strat = tf.distribute.MirroredStrategy(devices=["cpu:0", "cpu:1"]) with strat.scope(): m = MyModule() print(strat.run(m)) print("state:", m.state()) INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:CPU:0', '/job:localhost/replica:0/task:0/device:CPU:1') PerReplica:{ 0: tf.Tensor(-1.4154755, shape=(), dtype=float32), 1: tf.Tensor(-0.11388441, shape=(), dtype=float32) } state: tf.Tensor([256 0 0], shape=(3,), dtype=int64) with strat.scope(): tf.saved_model.save(m, filename) print("RNG stream from saving point:") print(strat.run(m)) print("state:", m.state()) print(strat.run(m)) print("state:", m.state()) INFO:tensorflow:Assets written to: ./saved_model/assets INFO:tensorflow:Assets written to: ./saved_model/assets RNG stream from saving point: PerReplica:{ 0: tf.Tensor(-0.68758255, shape=(), dtype=float32), 1: tf.Tensor(0.8084062, shape=(), dtype=float32) } state: tf.Tensor([512 0 0], shape=(3,), dtype=int64) PerReplica:{ 0: tf.Tensor(-0.27342677, shape=(), dtype=float32), 1: tf.Tensor(-0.53093255, shape=(), dtype=float32) } state: tf.Tensor([768 0 0], shape=(3,), dtype=int64) imported = tf.saved_model.load(filename) print("RNG stream from loading point:") print("state:", imported.state()) print(imported()) print("state:", imported.state()) print(imported()) print("state:", imported.state()) RNG stream from loading point: state: tf.Tensor([256 0 0], shape=(3,), dtype=int64) tf.Tensor(-1.0359411, shape=(), dtype=float32) state: tf.Tensor([512 0 0], shape=(3,), dtype=int64) tf.Tensor(-0.06425078, shape=(), dtype=float32) state: tf.Tensor([768 0 0], shape=(3,), dtype=int64) Uložiti SavedModel koji sadrži U distribucijsku strategiju se ne preporučuje jer će replike generirati isti tok slučajnih brojeva (što je zato što je ID replike zamrznut u SavedModelovom grafikonu). tf.random.Generator Ugradnja distribuiranog Stanje RNG-a će se ispravno vratiti, ali generirani slučajni brojevi bit će različiti od izvornog generatora u svojoj strategiji (još jednom zato što se uređaj izvan strategija tretira kao drugačiji od bilo koje replike u strategiji). tf.random.Generator Neovisni RNG Korištenje RNG-ova bez statusa je jednostavno.Budući da su to samo čiste funkcije, nema statusa ili nuspojava. print(tf.random.stateless_normal(shape=[2, 3], seed=[1, 2])) print(tf.random.stateless_normal(shape=[2, 3], seed=[1, 2])) tf.Tensor( [[ 0.5441101 0.20738031 0.07356433] [ 0.04643455 -1.30159 -0.95385665]], shape=(2, 3), dtype=float32) tf.Tensor( [[ 0.5441101 0.20738031 0.07356433] [ 0.04643455 -1.30159 -0.95385665]], shape=(2, 3), dtype=float32) Svaki RNG bez države zahtijeva Argument, koji mora biti cijeli Tensor oblika Rezultate op u potpunosti određuje ovo sjeme. seed [2] RNG algoritam koji se koristi za RNG bez statusa je ovisan o uređaju, što znači da isti algoritam koji radi na različitom uređaju može proizvesti različite izlaze. Algoritmi generala Obojica su Klase i funkcije podržavaju Philox algoritam (napisan kao ili na svim uređajima. tf.random.Generator stateless "philox" tf.random.Algorithm.PHILOX Različiti uređaji će generirati iste cijele brojeve, ako koriste isti algoritam i počevši od istog stanja. oni će također generirati "gotovo iste" brojeve plutajućih točaka, iako mogu postojati male numeričke razlike uzrokovane različitim načinima na koje uređaji izvode izračun plutajućih točaka (npr. redoslijed redukcije). XLA uređaji Na XLA uređajima (kao što je TPU, a također i CPU/GPU kada je XLA uključen) ThreeFry algoritam (napisan kao ili Ovaj algoritam je brz na TPU-u, ali spor na CPU/GPU-u u usporedbi s Philoxom. "threefry" tf.random.Algorithm.THREEFRY Ovaj papir Više informacija o ovim algoritmima. Paralelni slučajni brojevi: Jednostavno kao 1, 2, 3 Prvotno objavljen na web stranici TensorFlow, ovaj članak se ovdje pojavljuje pod novim naslovom i licenciran je pod CC BY 4.0. Prvotno objavljen na web stranici TensorFlow, ovaj članak se ovdje pojavljuje pod novim naslovom i licenciran je pod CC BY 4.0. TensorFlow TensorFlow