I-Python - i-Django: Akufanele Ungalokothi Ufake Lezi Zinto Ekuthengiseni

Okuqukethwe Ingxenye 1 Isifinyezo - Ngena Ngokujulile Kokwenziwe kwe-Django I-TLDR, Isifinyezo Ingabe okwenziwayo kungalimaza kanjani izinhlelo zethu zokusebenza? Ku Kuyingozi Phuma Okulandelayo - Imiyalo ye-PSQL ivimbana kanjani Imithombo Ingxenye 1 Isifinyezo - Ngena Ngokujulile Kokwenziwe kwe-Django kwangaphambilini , sifunde ukuthi kwenzekani uma umsebenzi ohlotshiswe nge ibizwa nangokuthi kwenzekani . Ngokufigqiwe: Kokuthunyelwe transaction.atomic with transaction.atomic() Uxhumano lwe-DB luyakhiwa noma luyabuyiswa. Kudalwa ukuthengiselana, isb, ithunyelwa ku-DB (uma i-DB kuyi-PSL noma okunye okuhlukile kwe-SQL). BEGIN; Kusukela manje kuze kube umsebenzi ukhona noma isitatimende esinesitatimende siphetha - kungaba ngephutha noma ngempumelelo - sizobe sisekwenziweni, futhi i-DB izobe isilindile. Lokhu kusho ukuthi (okungenani kuma-ACID DB afana ne-PSQL) okwenziwayo kuzobamba izingidi kumathebula nemigqa eshintshayo. Lapho kwenziwa umsebenzi we-Django DB, i-DB ibamba ukhiye ohambisanayo futhi ivimbele noma yikuphi ukusebenza okungqubuzanayo kulelo thebula noma irowu. Lokhu kungabangela okunye ukuxhumeka kuphelelwe yisikhathi ngenxa yokulinda ukukhishwa kwesikhiya. Uma ukusebenza kwehluleka noma uma kunephutha lesikhathi sokusebenza, i-DB ibuyisela emuva konke okwenziwayo bese ikhulula izingidi. Uma konke okwenziwayo kuphumelela, khona-ke zonke izinguquko ziyazibophezela futhi ziyatholakala kokunye ukuxhumana kwe-DB. Izingidi ziyakhululwa. Manje, sizoxoxa ngokuthi yini okufanele uyifake ku-transaction nokuthi yini okufanele uyigweme. Ngenxa yokuziphatha okulandelayo kokwenziwe, imisebenzi ethile iyingozi uma ibekwe kubhulokhi yokwenziwe. Okwenziwayo kubambelela ekukhiyeni kuze kube yilapho okwenziwayo kwehluleka noma sekuqediwe. Okwenziwayo kuhlehlisa yonke imisebenzi yabo ye-DB uma yehluleka. Isicelo sokwenziwe siyakhiya ngokushesha nje uma umsebenzi we-DB sewenziwe. I-TLDR, Isifinyezo Ku - Uma ukusebenza kwe-DB kungakwazi ukuhlehliswa futhi umsebenzi wehluleka, i-DB izoshiywa esimweni esibi ngoba umsebenzi uzozama ukubuyisela emuva izinguquko ngokuzenzakalelayo, kodwa azihlehliseki, ngakho-ke izohluleka. Imisebenzi ebuyiseleka emuva ku-DB - Asikwazi ukudala irekhodi elisha le-akhawunti yasebhange ngaphandle kwerekhodi lomsebenzisi, ngakho-ke sidinga ukudala kokubili ngomsebenzi ofanayo. Imisebenzi ehlobene ku-DB edingekayo - Ukubala ibhalansi isiyonke ikhasimende elinayo ngemva kokudala irekhodi elisha le-akhawunti ngediphozithi (ngisho nalokhu kungase kukhishwe kumsebenzi wokumodela okuhlakaniphile nokuxhumana). Ingqondo yebhizinisi ehlehliswayo nehlobene Kuyingozi - Ukulanda idatha kuvamise ukushesha, ngaphandle kwalezi zimo ezintathu. Le mibuzo izobambezela okwenziwayo futhi inwebise isikhathi lapho ibamba izikhiya, okuzoba nemiphumela emibi kwabanye abasebenzisi. Imibuzo enensa - Umsebenzi osebenza kumathebula amaningi ungakhiya ithebula ngalinye uze uqedwe. Lokhu kuvame kakhulu ekufudukeni kwe-Django - esinye isizathu sokugcina ukufuduka kuncane futhi kugxile etafuleni elilodwa noma ambalwa ngesikhathi. Ukusebenza kumathebula amaningi - Njengoba okwenziwayo kubambelela kukhiye kusukela lapho umbuzo wenziwe kuze kube yilapho umsebenzi wehluleka noma uphela, ukuthutha okusebenza kuyo yonke imigqa kuthebula kuzogcina kukhiya lonke ithebula, ngokuvimbela ukufunda noma ukubhala kumugqa ngamunye. Ukudluliswa Kwedatha (Okwe-PSQL ne-SQLite kuphela*) - lokhu kusebenza kudinga uhlobo oluqinile lokukhiya futhi, ngakho-ke, kuzovimbela ukufundwa/ukubhala kulo lonke ithebula. Le misebenzi iyona engase ibangele ukucisha. Ukushintsha amathebula noma amakholomu Phuma - Yonke into ekwenziweni kufanele ibuyisele emuva uma umsebenzi uhlehliswa; uma sifaka ikholi ye-API kumsebenzi, ayikwazi ukuhlehliswa. Imisebenzi engabuyiseleki emuva - Njengoba okwenziwayo kuvimbela yonke eminye imibuzo ukuthi isebenze kumathebula/amarowu okwenziwayo kuyashintsha, noma iyiphi ikhodi ekhulisa ubude besikhathi somsebenzi izobangela ukuthi i-DB ikhiywe, kubangele ukuphela kwesikhathi nokungaphenduli ezinhlelweni zokusebenza ezincike ku-DB. Ukuvimba amakholi Qhubeka ufunda ukuze uthole ezinye izindlela nezibonelo zamakhodi. Okwenziwayo Kungazilimaza Kanjani Izinhlelo Zethu Zokusebenza? Ingozi eyinhloko ukuthi okwenziwayo kubamba amalokhi kuze kube kwenziwa ukuze kuvinjelwe ukusebenza okungqubuzanayo kumathebula nemigqa futhi kuvumele okwenziwayo ukuthi kubuyiseleke emuva - lokhu kubalulekile ukwenza imisebenzi ye-DB kumsebenzi we-athomu. Lokhu kusho ukuthengiselana okuthatha isikhathi eside okusebenza kumathebula amaningi noma ambalwa abucayi kungase kubangele ukuphuma ngokuvala izingidi nokuvimbela ukufundwa/ukubhala kulawo mathebula/imigqa. Empeleni, uma sibeka ikhodi engalungile kubhulokhi ye-transaction, singakwazi ukwehlisa i-DB ngokuvimba konke okunye ukuxhumana ne-DB ekwenzeni imisebenzi kuyo. Ingcuphe yesibili ukuthi okwenziwayo kumele kubuyiselwe emuva futhi KULINDELEKE ukuthi kubuyiseleke emuva. I-DB ihlehlisa ngokuzenzakalelayo konke ukusebenza uma kwenzeka iphutha ekwenziweni. Ngakho-ke, imisebenzi ye-DB esiyifake ku-transaction kufanele ibuyiseleke emuva - ingxenye enkulu, asikho isidingo sokukhathazeka ngalokhu nge-PSQL. Kodwa kuthiwani ngamanye amakhodi? Ezikhathini eziningi, lapho sishintsha idatha yethu, sidinga ukwenza imisebenzi yokulandelela efana nokudubula imicimbi, ukubuyekeza amasevisi, ukuthumela izaziso zohlelo lokusebenza, njll. Le misebenzi AYIHAMBILEKI - asikwazi ukuhlehlisa umcimbi, isicelo, noma isaziso. Uma kwenzeka iphutha, izinguquko zedatha zibuyiselwa emuva, kodwa sesivele sithumele isaziso sohlelo lokusebenza sithi, "Umbiko wakho ukhiqiziwe; chofoza lapha ukuze uwubuke." Kwenzekani uma umsebenzisi noma amanye amasevisi enza lolu lwazi olungamanga? Kuzoba nenqwaba yokwehluleka. Ngakho-ke, noma iyiphi ikhodi engakwazi ukuhlehliswa akufanele ibe sekwenziweni, noma sizibeka engcupheni yokushiya isistimu yethu ikusimo esibi uma kwenzeka iphutha kumsebenzi. Ku - Uma ukusebenza kwe-DB kungakwazi ukuhlehliswa futhi umsebenzi wehluleka, i-DB izoshiywa esimweni esibi ngoba umsebenzi uzozama ukubuyisela emuva izinguquko ngokuzenzakalelayo, kodwa azihlehliseki, ngakho-ke izohluleka. Imisebenzi ebuyiseleka emuva ku-DB - Asikwazi ukudala irekhodi elisha le-akhawunti yasebhange ngaphandle kwerekhodi lomsebenzisi, ngakho-ke sidinga ukudala kokubili ngomsebenzi ofanayo. Imisebenzi ehlobene ku-DB edingekayo - Ukubala ibhalansi isiyonke ikhasimende elinayo ngemva kokudala irekhodi elisha le-akhawunti ngediphozithi (ngisho nalokhu kungase kukhishwe kumsebenzi wokumodela okuhlakaniphile nokuxhumana). Ingqondo yebhizinisi ehlehliswayo nehlobene Kuyingozi Lezi yizinto okuthi, kuye ngokuthi ingakanani idatha ecutshungulwayo kanye nethrafikhi ye-DB, engabangela ukucisha ngenxa yokubamba izingidi isikhathi eside kakhulu. Zonke lezi zinto zilungile uma zingathathi isikhathi eside. - Ukulanda idatha kuvame ukushesha, ngaphandle kwalezi zimo ezintathu. Le mibuzo izobambezela okwenziwayo futhi inwebise isikhathi lapho ibamba izikhiya, okuzoba nemiphumela emibi kwabanye abasebenzisi. Imibuzo enensa Izibonelo Imibuzo kumakholomu angekho inkomba Imibuzo ematafuleni amakhulu Uyajoyina @transaction.atomic def process_large_order_report(start_date, end_date, min_order_value=1000): # Complex query with multiple joins and aggregations large_orders = Order.objects.filter( created_at__range=(start_date, end_date), total_amount__gte=min_order_value, status='completed' ).select_related( 'customer', 'shipping_address', 'billing_address' ).prefetch_related( 'items__product__category', 'items__product__supplier' ).annotate( item_count=Count('items'), total_weight=Sum('items__product__weight'), discount_percentage=F('discount_amount') * 100 / F('total_amount') ).filter( # Additional complex filtering Q(customer__user__is_active=True) & (Q(items__product__category__name='Electronics') | Q(items__product__category__name='Furniture')) & ~Q(shipping_address__country='US') ).order_by('-total_amount') # do the transactional work with the large_orders queryset Okunye Yenza le mibuzo ngaphambi nangaphandle kokuthenga. # fixed def process_large_order_report(start_date, end_date, min_order_value=1000): # Complex query with multiple joins and aggregations large_orders = Order.objects.filter( created_at__range=(start_date, end_date), total_amount__gte=min_order_value, status='completed' ).select_related( 'customer', 'shipping_address', 'billing_address' ).prefetch_related( 'items__product__category', 'items__product__supplier' ).annotate( item_count=Count('items'), total_weight=Sum('items__product__weight'), discount_percentage=F('discount_amount') * 100 / F('total_amount') ).filter( # Additional complex filtering Q(customer__user__is_active=True) & (Q(items__product__category__name='Electronics') | Q(items__product__category__name='Furniture')) & ~Q(shipping_address__country='US') ).order_by('-total_amount') # Start the transaction block with transaction.atomic(): # do the transactional work with the large_orders queryset — umsebenzi osebenza kumathebula amaningi ungakhiya ithebula ngalinye uze uqedwe. Lokhu kuvame kakhulu ekufudukeni kwe-Django - esinye isizathu sokugcina ukufuduka kukuncane futhi kugxile etafuleni elilodwa noma ambalwa ngesikhathi. Ukusebenza kumathebula amaningi Izibonelo Bheka Ukushintsha ithebula noma isakhiwo sekholomu class Migration(migrations.Migration): dependencies = [("migrations", "0001_initial")] # too many operations operations = [ migrations.RemoveField("Author", "age"), migrations.AddField("Author", "rating", models.IntegerField(default=0)), migrations.AlterField("Book", "price", models.DecimalField(max_digits=5, decimal_places=2)), ] Okunye Hlukanisa okwenziwayo wenze imisebenzi emincane eminingi futhi uyibophe ku-oncommit callbacks. # fixed # 1st migration class Migration(migrations.Migration): dependencies = [("migrations", "0001_initial")] operations = [ migrations.RemoveField("Author", "age"), ] # 2nd migration class Migration(migrations.Migration): dependencies = [("migrations", "0002_initial")] operations = [ migrations.AddField("Author", "rating", models.IntegerField(default=0)), ] # 3rd migration class Migration(migrations.Migration): dependencies = [("migrations", "0003_initial")] operations = [ migrations.AlterField("Book", "price", models.DecimalField(max_digits=5, decimal_places=2)), ] - Njengoba okwenziwayo kubambelela kukhiye kusukela lapho umbuzo wenziwe kuze kube yilapho okwenziwayo kwehluleka noma ukuphela, ukufuduka okusebenza kuyo yonke imigqa yetafula kuzogcina kukhiya lonke itafula, noma ngokuvimbela ukufunda noma ukubhala kumugqa ngamunye. Ukufuduka Kwedatha def migrate_user_profiles(): # Get all users with legacy profiles users_with_profiles = User.objects.filter( legacy_profile__isnull=False ).select_related('legacy_profile') # Process all users in a single transaction with transaction.atomic(): # Track progress total = users_with_profiles.count() print(f"Migrating {total} user profiles...") # Process each user for i, user in enumerate(users_with_profiles): if i % 100 == 0: print(f"Processed {i}/{total} profiles") legacy = user.legacy_profile legacy.update_new_user_profile() Okunye Songa okwenziwayo eduze kwemisebenzi ngayinye, hhayi konke ukufuduka. Ngokubeka izibuyekezo kumugqa ngamunye ekwenziweni, sibamba kuphela izingidi isikhathi esifushane. # fixed def migrate_user_profiles(): # Get all users with legacy profiles users_with_profiles = User.objects.filter( legacy_profile__isnull=False ).select_related('legacy_profile') # Process all users in a single transaction # Track progress total = users_with_profiles.count() print(f"Migrating {total} user profiles...") # Process each user for i, user in enumerate(users_with_profiles): if i % 100 == 0: print(f"Processed {i}/{total} profiles") with transaction.atomic(): legacy = user.legacy_profile legacy.update_new_user_profile() (Okwe-PSQL ne-SQLite *) — le misebenzi idinga uhlobo oluqine kakhulu lokukhiya, ngakho-ke, izovimbela ukufundwa/ukubhala kulo lonke ithebula. Le misebenzi iyona engase ibangele ukucisha. kuphela Ukushintsha amathebula noma amakholomu Isibonelo Shintsha Ikholomu Shintsha Ithebula Okunye Zigijime kamuva. Le mibuzo iyadingeka, KODWA Akudingekile ukuthi siyiqhube ngamahora ebhizinisi. Inqubomgomo engcono kakhulu lapha ukunciphisa ubungozi bokunqamuka ngokunquma ukuthi itafula libaluleke kangakanani, ukulinganisa ukuthi ukufuduka kungathatha isikhathi esingakanani, ukuqalisa ukuthutha lapho i-DB inethrafikhi encane, kanye nokulungiselela uhlelo lokuhlehlisa. Nciphisa isikhathi osithathayo. Lokhu kungenziwa ngokuhlukanisa ithebula nokusebenzisa ukufuduka kuma-partitions angawodwana. PSQL Partitions & Django *I-Django igoqa kuphela okwenziwayo mayelana nokufuduka kwe-PSQL ne-SQLite. class Migration(migrations.Migration): dependencies = [("migrations", "0001_initial")] # this migration, if on a large table, can slow down and block other operations # do it later operations = [ migrations.RemoveField("Users", "middle_name"), ] Phuma — Yonke into ekwenziweni kufanele ibuyisele emuva uma umsebenzi uhlehliswa; uma sifaka ikholi ye-API kumsebenzi, ayikwazi ukuhlehliswa. Imisebenzi engabuyiseleki emuva Izibonelo Ukwengeza umcimbi kulayini — qala ezinye izehlakalo Ukuthumela Ucingo lwe-API — izibuyekezo zesimo, vula imisebenzi, njll. def transaction(user_data, user_files): with transaction.atomic(): user = User.objects.create(**user_data) async_notification_service.send_email(user.email, "You can login now!") Account.objects.create(user=user, balance=0) # rest of user creation proccess Ezinye izindlela Yenza umqondo webhizinisi obalulekile odinga ukwenzeka ekushayeleni emuva kwe-oncommit - I-oncommit callback IHLALA ibizwa ngokuthi ngemva kokuba umsebenzi usuphumelele, futhi zonke izibuyekezo zokwenziwe ziyatholakala ku-oncommit callback. Singakwazi futhi ukwenza umsebenzi ubuyiseleke emuva, okungukuthi, sidale indlela yokususa imicimbi noma ukuthumela ikholi ye-API — Lona umsebenzi ongabalulekile wethimba eliphethe ulayini womcimbi noma i-API. Angikuncomi. def transaction(user_data, user_files): with transaction.atomic(): user = User.objects.create(**user_data) Account.objects.create(user=user, balance=0) # rest of user creation proccess # the transaction is still in progress, so it can still be rolled back, it is not # committed until the transaction block is exited, so putting the notification here # is not a good idea - especially if the job starts immediately tries to read the data # this creates a race condition async_notification_service.send_email(user.email, "You can login now!") def transaction(user_data, user_files): with transaction.atomic(): user = User.objects.create(**user_data) Account.objects.create(user=user, balance=0) # rest of user creation proccess transaction.on_commit(partial(async_notification_service.send_email, user.email, "You can login now!")) — njengoba okwenziwayo kuvimbela yonke eminye imibuzo ukuthi isebenze kumathebula/amarowu okwenziwayo kuyashintsha, noma iyiphi ikhodi ekhulisa ubude besikhathi somsebenzi izobangela ukuthi i-DB ivalwe, kubangele ukuphela kwesikhathi nokungaphenduli ezinhlelweni zokusebenza ezincike ku-DB. Ukuvimba izingcingo Izibonelo Izingcingo zenethiwekhi - Ukuthumela izicelo kuma-API ukuze uthole idatha ezofakwa ekuthengiseni. Ukugawulwa kwemithi kungaphansi kwalokhu — Kuye ngokuthi umgawuli esiwusebenzisayo, umtapo wezincwadi ungase uthumele izingodo lapho sisethengisweni, noma uma sisebenzisa i-datadog, izoba nenye inqubo yokulondoloza izingodo, kodwa sisakhokha intengo yokugcina izingodo kumemori kuze kube yilapho umsebenzi weqoqo uwagcina efayeleni. Ukusebenza kwediski - Ilayisha i-CSV ukuze ifakwe etafuleni noma ukuthumela itafula ku-CSV. Imisebenzi enzima ye-CPU - Ukuphindaphinda kwe-Matrix noma noma yikuphi ukuguqulwa kwezibalo/idatha okusindayo kuzovimba i-CPU nayo yonke eminye imisebenzi - I-Python's Global Interpreter Lock iphoqa yonke imicu ukuthi isebenzise i-CPU ngesikhathi esisodwa, futhi i-Django icutshungulwa kanye (konke ukusebenza kokwenziwe kwenzeka ngokulandelana). def transaction(user_data, user_files): with transaction.atomic(): user = User.objects.create(**user_data) for file_data in user_files: # transaction waits for this upload and so do all other connections that need access to table/rows the transaction # uses url = Cloudinary.upload_file(file_data['data']) Files.objects.create(**file_data['meta_data'], user=user, url=url) Account.objects.create(user=user, balance=0) # rest of user creation proccess Ezinye izindlela Lungisa idatha ngaphambi kokuthenga — Layisha idatha yediski/inethiwekhi kusengaphambili. Uma idatha isilungile, yenza umsebenzi ngayo. Sebenzisa idatha yesimeli - Dala isethi yedatha yesimeli esisobala (ayikho indlela yokuyenza iphutha ngedatha yokukhiqiza) ebuyiseleka kalula futhi ibonakale. Ilayishe ngaphambi kokuthenga futhi uyisebenzise. Khiqiza idatha - Uma kunezingqinamba ezihlukile ezinkundleni ezivimbela idatha yesimeli ukuthi isetshenziswe. Lokhu kuyingozi - Kuzoholela ekuhlulekeni okungalindelekile ngenxa yemvelo yama-algorithms angahleliwe - ngakho-ke sebenzisa i-algorithm enhle engahleliwe enembewu enhle ukuze ugweme ukwehluleka okungalindelekile. Susa noma uguqule izithiyo — Uma izithiyo zingasivumeli ukuthi sakhe amarekhodi ethu ngokuphepha, kunenkinga nge-schema yethu. Yenza izithiyo zincike kukholomu yezwe ecacisa ukuthi irekhodi seliqediwe nini futhi kufanele liqashwe. # not bad def transaction(user_data, user_files): user = None with transaction.atomic(): user = User.objects.create(**user_data) Account.objects.create(user=user, balance=0) # rest of user creation proccess for file_data in user_files: url = Cloudinary.upload_file(file_data['data']) Files.objects.create(**file_data['meta_data'], user=user, url=url) # best fix from functools import partial def transaction(user_data, user_files): user = None with transaction.atomic(): user = User.objects.create(**user_data) Account.objects.create(user=user, balance=0) # rest of user creation proccess # partials create a callable with the function and arguments # so that the function is called with the arguments when the transaction is committed # TODO: diff between partial and lambda here??? transaction.on_commit(partial(create_user_files, user_files, user)) def create_user_files(user_files, user): for file_data in user_files: url = Cloudinary.upload_file(file_data['data']) Files.objects.create(**file_data['meta_data'], user=user, url=url) Okulandelayo - Imiyalo ye-PSQL Ivimbana Kanjani Eposini elilandelayo, sizongena ku-PSQL futhi sithole: Ihluke kanjani imiyalo yokukhiya amathebula nemigqa Imiphi imiyalo eyingozi kakhulu ukuyenza Imithombo Athomu Ukufuduka kwe-Django & Transactions Kumininingwane yolwazi esekela okwenziwayo kwe-DDL (i-SQLite ne-PostgreSQL), yonke imisebenzi yokuthutha izosebenza ngaphakathi kokwenziwe okukodwa ngokuzenzakalelayo. Ngokuphambene, uma isizindalwazi singakusekeli okwenziwayo kwe-DDL (isb. MySQL, Oracle) yonke imisebenzi izosebenza ngaphandle kokwenziwe. Unganqanda ukuthuthela kwelinye izwe kusukela ekusebenzeni ngokusetha isibaluli se-athomu sibe Amanga. Ngokwesibonelo: Kuyenzeka futhi wenze izingxenye zokufuduka ngaphakathi kokuthenga usebenzisa i-atomic() noma ngokudlula i-atomic=True to RunPython PSQL Partitions & Django I-ORM ka-Django ayinakho ukusekela okwakhelwe ngaphakathi kwamathebula ahlukanisiwe, ngakho-ke uma ufuna ukusebenzisa ama-partitions kuhlelo lwakho lokusebenza, kuzothatha umsebenzi omncane owengeziwe. Enye indlela yokusebenzisa ama-partitions ukugoqa okwakho ukufuduka okusebenzisa i-SQL eluhlaza. Lokhu kuzosebenza, kodwa kusho ukuthi kuzodingeka ukuthi ulawule ukuthuthela kwelinye izwe kuzo zonke izinguquko ozenzayo kuthebula esikhathini esizayo. Enye inketho ukusebenzisa iphakheji ebizwa ngokuthi i-django-postgres-extra. I-Django-postgres-extra inikeza ukusekelwa kwezici ezimbalwa ze-PostgreSQL ezingakhelwanga ku-ORM ka-Django, isibonelo, ukusekelwa kwe-TRUNCATE TABLE nokuhlukaniswa kwetafula. PSQL Transactions I-Python: Isebenza Kanjani I-Django Transactions