D. Dvilinskas and J. Lasevičius received the invitation to join together with the founders of Appzone, CoverManager, HarborLab, Immensa and LogComex. "This global network not only brings together a community of founders but also commits to investing in ...

TransferGo Founders Become the First Lithuanians to be Invited to the Global Entrepreneur Network, Endeavor

En un mundo en constante transformación, no se trata de una lucha entre la tecnología y los humanos o entre lo tradicional y lo nuevo, sino en saber alinearnos y potenciarnos

Javier Carrizo

LogComex

logcomex

In 2021, hosted the first-ever awards!! HackerNoon Startup of the Year The goal of this campaign was to give recognition to the hardworking startups who pulled through the pandemic -- growing and thriving (despite all of the challenges). More than 44,000 startups were nominated all across the globe! They were selected from over 4,553 cities, with a total number of * cough * “4 billion” votes. Insane, right? To see who is a winner in each respective city, visit and pick a city. The name at the top of the list is the winner. (Awards, with compromised reviews, are under review.) startups.hackernoon.com The will announce the winners per region in their respective cities. Startups Winners Announcement Series Apparently, South America startups did incredibly well in that regard. With so many, until now, uncrowned jewels, it’s time we present to you... * drumroll* 🥳 The South America Startups of the Year Winners! Santiago, Chile Chile is known for being the most stable country to do business in Latin America, ranking 13th in The Economist’s Business Environment Index. Moreover, according to the Global Innovation Index, Chile ranks first in Latin America for innovation and is recognised as Chilean Silicon Valley with Santiago being the central city. The winner is…. QCObjects It is the 1st world class open-source framework made in Latin-America. São Paulo, Brazil And the winner of Brazil’s vibrant financial center is... Innvo Digital Their mission is to create the best ecosystem for a technology professional to work and undertake while sharing it with our customers and society. São Carlos The absolute leader of Sao Carlos seems to be: Predify Predify is an online software for optimizing pricing proccess and improving profits with AI and Big Data Analytics. Belo Horizonte, Brazil With a more than noble goal, Belo Horizone’s winner is: CM Tecnologia CM Tecnologia is an online scheduling system for healthcare providers in Brazil. Porto Alegre, Brazil With the focus on data protection, Porto Alegre, Brazil’s winner is…. Privacy Tools Privacy compliance platform, based on AI/Blockchain, which helps global companies to keep compliant with the data protection requirements. Brasil, Brazil With the focus on EdTech, Brasil is bringing us: Awari Awari is a YC-W21 company focusing on education for the Tech Industry. The company is growing at a fast pace and has gathered experts from all leading tech companies into a full education experience to prepare the next cohort of tech talent. Quito The capital of Ecuador, and also the closest capital to the actual equator, as the winner brings us: Agrodemanda The most comprehensive agricultural directory in Ecuador. Rio De Janeiro, Brazil Rio de Janeiro has more than 300 startups and is ranked 3rd in Brazil in terms of startup ecosystem development. The city is famous for the startups in the spheres of education, energy & environment and foodtech. Their winner is…. Bybrand Bybrand is the simplest way to save time and avoid frustration when managing your employee's email signatures. Buenos Aires, Argentina Buenos Aires, Argentina is not only the cosmopolitan capital with a vibrant history. It’s also Latin America’s most important port. Which makes their winner actually a very logical choice. Please meet: Liftit Liftit is the leading technology platform to automate and execute last-mile truck deliveries across Latin America. Curitiba, Brazil With a more noble mission and sustainability in mind, Curitiba, Brazil, as the winner brings us... Discrepante is the marketplace for a more human and sustainable economy Discrepante.com Florianópolis, Brazil Would you really be surprised one of the Brazil’s winners is related to football training? We neither. Please meet... Fútbol Fit Club Football FitClub® is a fitness training firm that gives fitness training to improve like a professional football player. Medellín, Colombia And the first Colombian Startup Winner is is a data intelligence solution powered by a high quality team. Meet... LogComex Market intelligence for international trade in Latin America Montevideo, Uruguay The host of inaugural FIFA world cup of 1930 comes through when it comes to innovation even in 2021. I’d like to introduce you to: Drop Technologies First electronic hand antiperspirant device Lima, Peru Lima, Peru, has in 2021 delivered some amazing results when it comes to blockchain technology. Their winner is very well deserved: RSK Labs RSK is the first open-source smart contract platform São Carlos Do Pinhal Well this is a unique case. The absolute unanimous winner of São Carlos Do Pinhal is: brain4care brain4care solution enables monitoring intracranial compliance noninvasively anywhere Blumenau, Brazil We’re sensing a theme here. Another stellar EdTech winner is coming from Blumenau, Brazil. Please meet... Uniasselvi Uniasselvi is a Brazilian distance-learning company that offers more than 60 undergraduate and postgraduate courses, in person and distance. Bogotá, Colombia Startups of Bogotá, Colombia definitely showed they’re not to be underestimated. With a very noble mission, let me introduce you to: Risü Risü is an online platform that links online consumption to donation to social institutions. Campina, Brazil If there’s one typo of business we’d single out as taking the biggest hit out of the pandemic, it would have to be small businesses. Thankfully, the winner of Campina, Brazil, operates with small businesses in mind Superlógica Tecnologias Ltda. Superlogica is a management system company created for small service businesses. Cerqueira César, Brazil Swooping half the points from the entire City, this year’s winners of Cerqueira César are InfoPrice INFOPRICE is a business intelligence company focused on physical retail pricing. Governador Valadares, Brazil The same can be said for the winners here. Let’s hear it for: Grupo KPG Saas, Payment, Manufacture, Pharmaceutical, E-commerce, Marketplace, Logistics Caxias Do Sul Known for wine making and Italian cultural influence, Caxias Do Sul brings as an interesting winner. Studio 360 Studio 360 is a proptech focused on easing the purchase experience for real state companies, brokers, and costumers. Sorocaba From Sorocaba, we have an interesting concept coming. Looking forward to seeing it further develop. Let’s meet: Quinto Quinto is a social network that represents the collective opinion of society—to help democracy evolve through technology Viña Del Mar To be a venue-oriented business and survive through the pandemic definitely deserves some recognition. In the name of that, let’s meet: SonicPollen SonicPollen, based in Chile, is a web-based booking platform that connects bands with venues and organizers. Lages Learn more about high-growth startups in Lages, South America. and the winner is…. Qualiall Qualiall develops technologies for training in operational processes such as work security, equipment maintenance, and more. Córdoba, Argentina Another gem from Argentina brings us unique agricultural solutions. Please meet: Kilimo Kilimo is a Big Data solution for water management in agriculture. São José Dos Campos, Brazil Another great agricultural solution comes this time from Brasil. Meet São José Dos Campos Startup winner: TerraMagna TerraMagna is a fintech company that focuses on risk mitigation for agribusinesses. São Leopoldo, Brazil Event-oriented businesses that have survived in the previous years should definitely bottle their perseverance and start selling it. One of those is definitely this winner: Wedy Wedy offers a one-stop-social marketplace for weddings. Pelotas, Brazil Medical scheduling platforms are definitely something we need to see more of. So glad Pelotos winner comes from this particular niche. Fácil Consulta Fácil Consulta is a booking platform that allows people to schedule an appointment with a Specialist Physician in their area. Santa Maria Could you believe how versatile the startups of South America are when we are only now coming across a game development studio? Let’s meet them Imgnation Studios Imgnation Studios is a game development studio that focuses on virtual reality. Asunción, Paraguay A FinTech solution coming from Paraguay. Meet Asunción Startup of the Year winner Vision Banco Vision Banco is a solid and growing financial institution serving the financial needs of the micro, small and medium enterprises. Caracas Venezualan food delivery app that seemed to have your heart. Meet Caracas Startup of the Year winner Yummy Inc. First Food Delivery App in Venezuela Recife A suitable combo between gaming and education, Recife Startup of the Year winner brings revolutionary solutions. Let’s meet them. Joy Street Joy Street offers develops video games focused on educational needs. Rosario, Argentina The winner of Rosario Startup of the Year award for 2021 is... Gbot Gbot is an autonomous robot that controls weeds without using agrochemicals. We also collect data in fields and generate precision maps. Vila Olímpia, Brazil The winner of Vila Olímpia Startup of the Year award for 2021 is... Seekr Seekr operates in the Information Technology sector, providing tools for managing and monitoring of brands. Barueri The winner of Barueri Startup of the Year award for 2021 is.... Glic Brazilian mhealth (web/app) platform for diabetes management Brasília The winner of Brasília Startup of the Year award for 2021 is... Ticies Ticies is a social network that helps people plan where to go with their friends, relatives and acquaintances, in all cities of the world. Juiz De Fora The winner of Juiz De Fora Startup of the Year award for 2021 is... Qranio Qranio is a gamified mobile learning platform for enterprises that rewards learning and stimulates users to engage with educational content. La Paz The winner of La Paz Startup of the Year award for 2021 is... Jukumari Advice 360 photo based market place for responsible tour operators Temuco The winner of Temuco Startup of the Year award for 2021 is... GoFeels GoFeels is the first independent hospitality network in the world. We promote collaborative work among hotels around the world. Vitacura The winner of Vitacura Startup of the Year award for 2021 is... MeatMe MeatMe delivers meats and accessories to have a culinary experience. Barranquilla The winner of Barranquilla Startup of the Year award for 2021 is... Mandaos Mandaos is a digital platform that connects companies and people with a network of independent messengers Ceará The winner of Ceará Startup of the Year award for 2021 is... Caroá Caroá is a cooperative that sells 34 varieties of vegetables, organic fruits, and vegetables. Chapecó The winner of Chapecó Startup of the Year award for 2021 is... HortiAgro HortiAgro is a seed and fertilizer market. Chile Chico The winner of Chile Chico Startup of the Year award for 2021 is... Gleengo Gleengo connects children with their favorite toys, through a tiny device and an application based on text-to-voice. Goiânia The winner of Goiânia Startup of the Year award for 2021 is... Auvo Auvo is an external teams management software. Itá The winner of Itá Startup of the Year award for 2021 is... TISSUELAB Tissuelab is a research and development center focused on biotechnology, and systems for reconstructive musculoskeletal surgery. Jardim Paulista The winner of Jardim Paulista Startup of the Year award for 2021 is... DiviCom DiviCom is a digital video networking company that designs, integrates, and markets complete systems. João Pessoa The winner of João Pessoa Startup of the Year award for 2021 is... YupiPlay YupiPlay develops and offers an educational gaming platform. La Reina The winner of La Reina Startup of the Year award for 2021 is... INTI-TECH INTI-TECH is the fastest growing renewable energy in Chile and the world's leading unconventional source. Leme The winner of Leme Startup of the Year award for 2021 is... EcoStart EcoStart focuses on manufacturing sustainable electric bicycles, fans, and accessories. Loja The winner of Loja Startup of the Year award for 2021 is... Enerwi Corporation Keeping people connected and secured through wireless charging novel solutions Manizales The winner of Manizales Startup of the Year award for 2021 is... Porter Metrics Porter Metrics has developed an automated solution for generating social media reports in minutes. Nova Lima The winner of Nova Lima Startup of the Year award for 2021 is... Biomimetic Solutions Biomimetic Solutions is an innovative tissue engineering company. Paraná The winner of Paraná Startup of the Year award for 2021 is... Laura Laura is a provider of artificial intelligence and predictive analytics platform for hospitala and clinics to prevent damage & reduce costs. Salvador The winner of Salvador Startup of the Year award for 2021 is... Sanar Sanar is a platform that focuses on being the doctor's guide and support, offering specialized content that helps in their development. Santa Catarina The winner of Santa Catarina Startup of the Year award for 2021 is... DrPOCKET Sunchales The winner of Sunchales Startup of the Year award for 2021 is... Eolo Pharma Eolo Pharma focuses on developing drugs targeting the molecular and cellular basis of inflammation-related diseases Valparaiso The winner of Valparaiso Startup of the Year award for 2021 is... Invitta Invitta transforms plastic waste into a high-performance construction material for manufacturing sustainable urban furniture. Vila Velha The winner of Vila Velha Startup of the Year award for 2021 is... OneGrid OneGrid is a developer of hardware and software solutions focused on energy management. Indaiatuba The winner of Indaiatuba Startup of the Year award for 2021 is... DWUL Energy Renewable Energy Campinas, Brazil The winner of Campinas Startup of the Year award for 2021 is... Ativy Ativy is an IT company specialized in cloud computing, data centers, IT outsourcing, service desk, and IT management. Las Condes, Chile The winner of Las Condes Startup of the Year award for 2021 is... # MuvPass Subscription services that allows it's users unlimited access to fitness classes in the best gyms and studios in Latin America. Pinheiros, Brazil The winner of Pinheiros Startup of the Year award for 2021 is... GuiaBolso GuiaBolso is a personal finance platform that empowers people to take control of financial well-being. Rio, Brazil The winner of Rio Startup of the Year award for 2021 is... EmCasa EmCasa is a vertically-integrated platform that helps customers buy and sell a home in Latin America, starting in Brazil. Guayaquil The winner of Guayaquil Startup of the Year award for 2021 is... Sabroso Online platform for LatAm's enterprises to order low-cost meals Jundiaí The winner of Jundiaí Startup of the Year award for 2021 is... Sintecsys Sintecsys offers a fast and effective detection, reducing losses and costs with fire brigades. Minas Gerais The winner of Minas Gerais Startup of the Year award for 2021 is... Rizoflora Biotechnology Rizoflora Biotechnology is a producer of biological product designed for nematodes control. São Caetano Do Sul The winner of São Caetano Do Sul Startup of the Year award for 2021 is... Wevo 1 iPaaS in Latam - Enabling companies to be truly digital companies with APIs, systems and data integrated into a single platform Canoas The winner of Canoas Startup of the Year award for 2021 is... ContaÁgil ContaÁgil facilitates and reduces bureaucracy in accounting routines of companies. Chile The winner of Chile Startup of the Year award for 2021 is... Unight Unight is the future of nightlife social media. Israelândia The winner of Israelândia Startup of the Year award for 2021 is... Matchpool Matchpool is a platform that creates human connections. La Plata The winner of La Plata Startup of the Year award for 2021 is... Pixivus Pixivus offers a community-based application that enables its users to collect, store, and share pictures of events. Novo Hamburgo The winner of Novo Hamburgo Startup of the Year award for 2021 is... Devorando Devorando offers an online food ordering system and menus available in restaurants based in Brazil. Pernambuco The winner of Pernambuco Startup of the Year award for 2021 is... Redu Redu is a distant learning platform which is built on a social networking structure. Providencia The winner of Providencia Startup of the Year award for 2021 is... # TrainFES TrainFES is a medical Devices for the rehabilitation of people with paralysis. Ribeirão Prêto The winner of Ribeirão Prêto Startup of the Year award for 2021 is... AllyHub Ally is a sales distribution channel technology that connects Schools and Service Providers with Education Agencies. Santa Cruz De La Sierra The winner of Santa Cruz De La Sierra Startup of the Year award for 2021 is... Mobi Latam Mobi Latam is a clean energy startup that focuses on transportation systems based on electric mobility. Santana De Parnaíba The winner of Santana De Parnaíba Startup of the Year award for 2021 is... MercadoLivre.com is an e-commerce technology company providing solutions to individuals and companies buying, selling, and more online. MercandoLivre.com More Information If you’re a winner, please answer ! If you’re a reader, participant, just a curious individual, consider answering one of our interviews . this interview here Read the FAQs about or . Startups here here To see all of the winners you may visit , pick any city and view the name that appears at the top. this site If you’re interested in partnering with us, we invite you to look through some of these or reach out to our business department. unique sponsorship options Perhaps creating your very own tickles your fancy or maybe you have a whole new idea for how we can partner! writing contest Thank you to all of our sponsors! We truly appreciate your contribution to the tech world and thus the future! The Startups Awards are sponsored by: , , , , , , , , , , Blockster Brex FreeCodeCamp LaunchDarkly Coil Udacity Udemy Vercel Asia Pacific Transgender Network Bybit Everscale Network This post was written/compiled by Shreya Kundu and Katarina Andrejević! If you have any questions or concerns or would like to report a discrepancy please email ellen@hackernoon.com.

Meet the Winners of Startups in South America

A Deeper Dive Into Log Enrichment

লগ সমৃদ্ধকরণে একটি গভীর ডুব

Ein tieferer Einblick in die Protokollanreicherung

Una inmersión más profunda en el enriquecimiento de registros

Une plongée plus approfondie dans l'enrichissement des journaux

लॉग संवर्धन में एक गहरा गोता

ログの強化についての詳細

로그 강화에 대한 심층 분석

Um mergulho mais profundo no enriquecimento de toras

Более глубокое погружение в обогащение журналов

Günlük Zenginleştirmeye Daha Derin Bir Bakış

Đi sâu hơn vào việc làm giàu nhật ký

深入研究日志丰富

Logging in Azure with Application Insights and Serilog

What I learned by migrating an RDS database into production using AWS DMS (Database Migration Service) …or how to avoid a big headache while live migrating and replicating your db instances I am currently managing the migration of some on premise services, and other services already present on AWS under a single VPC. Having active production systems H24 7/7 it was essential to design and implement a fairly precise migration plan with a zero error margin. While regarding services like EC2 instances, EFS, Security Group, Target Group and Load Balancer the transition was quite simple, I encountered some obstacles during the migration of our RDS database (MariaDB) from one VPC to another. For this task the native Amazon service, DMS (Database Migration Service) was chosen. Obviously, not being able to allow discrepancies between the existing and new production databases, the obligatory choice fell on a "live" migration through the "Ongoing replication" mode. According to Amazon: “…the quickest and easiest way to migrate databases to AWS with low cost and minimal downtime.” Even if i managed to successfully migrate other non-production databases using DMS, i was kinda like… <a href="https://medium.com/media/02ed01f042aa15adced4ad548395d86a/href">https://medium.com/media/02ed01f042aa15adced4ad548395d86a/href</a> To avoid unpleasant surprises, I performed a snapshot of the production database, created a new RDS instance (mariaDB) to then restore the snapshot on the new database. For those who have never used DMS, the service involves the creation of a (source and target) and a (which will take care of migrating data from one database to another).I preferred to use the db of the production db, and not of another database, as I wanted to replicate in “sandbox env” the exact structure of the database, including the attributes of the tables. replication instance, two endpoints task In this article we will not deal with the technical aspects, configurations related to DMS, so if you are already accustomed the following represents the replication task created for my prod-test migration (ARNs are not visible cause … you know why… ): Task name 
An identifier for your Task

***********:-db-task

Task ARN
This ARN is the stable way uniquely identify your Replication Task when calling DMS APIs

arn:aws:dms:eu-central-1:***********:task:***************

Status
The current computed status of the task. Note that this is a computed value and may not match the raw status from the service API

Load complete

Migration type
How should this task migrate data

Full Load

Replication instance

:***********:

Source endpointSource endpoint

:***********:-frankfurt

Target endpointTarget endpoint

:***********:-db

Mapping method
This is a json document that details how source tables are mapped to the target

{"rules":[{"rule-type":"selection","rule-id":"1","rule-name":"1","object-locator":{"schema-name":"tmp","table-name":"%"},"rule-action":"include","filters":[]},{"rule-type":"selection","rule-id":"2","rule-name":"2","object-locator":{"schema-name":"digital_content_store_kareem","table-name":"%"},"rule-action":"include","filters":[]},{"rule-type":"selection","rule-id":"3","rule-name":"3","object-locator":{"schema-name":"digital_content_store","table-name":"%"},"rule-action":"include","filters":[]},{"rule-type":"selection","rule-id":"4","rule-name":"4","object-locator":{"schema-name":"dcw_unit_testing","table-name":"%"},"rule-action":"include","filters":[]},{"rule-type":"selection","rule-id":"5","rule-name":"5","object-locator":{"schema-name":"dcs_unit_testing","table-name":"%"},"rule-action":"include","filters":[]},{"rule-type":"selection","rule-id":"6","rule-name":"6","object-locator":{"schema-name":"dcs_issam","table-name":"%"},"rule-action":"include","filters":[]},{"rule-type":"selection","rule-id":"7","rule-name":"7","object-locator":{"schema-name":"DCW_reporting","table-name":"%"},"rule-action":"include","filters":[]},{"rule-type":"selection","rule-id":"8","rule-name":"8","object-locator":{"schema-name":"DCW","table-name":"%"},"rule-action":"include","filters":[]}]}

Last failure message
The last failure message from the time the task was run and failed

Created

March 28, 2019 at 4:11:15 PM UTC+1

Started

March 29, 2019 at 2:14:56 PM UTC+1

CDC start positionOption to start Change Data Capture (CDC)

CDC stop positionOption to stop Change Data Capture (CDC)

CDC recovery checkpoint
DMS specific recovery checkpoint string to recover from a previous point

Task Settings
{
  "TargetMetadata": {
    "TargetSchema": "",
    "SupportLobs": true,
    "FullLobMode": true,
    "LobChunkSize": 64,
    "LimitedSizeLobMode": false,
    "LobMaxSize": 0,
    "InlineLobMaxSize": 0,
    "LoadMaxFileSize": 0,
    "ParallelLoadThreads": 0,
    "ParallelLoadBufferSize": 0,
    "BatchApplyEnabled": false,
    "TaskRecoveryTableEnabled": false
  },
  "FullLoadSettings": {
    "TargetTablePrepMode": "DROP_AND_CREATE",
    "CreatePkAfterFullLoad": false,
    "StopTaskCachedChangesApplied": false,
    "StopTaskCachedChangesNotApplied": false,
    "MaxFullLoadSubTasks": 8,
    "TransactionConsistencyTimeout": 600,
    "CommitRate": 10000
  },
  "Logging": {
    "EnableLogging": false,
    "LogComponents": [
      {
        "Id": "SOURCE_UNLOAD",
        "Severity": "LOGGER_SEVERITY_DEFAULT"
      },
      {
        "Id": "SOURCE_CAPTURE",
        "Severity": "LOGGER_SEVERITY_DEFAULT"
      },
      {
        "Id": "TARGET_LOAD",
        "Severity": "LOGGER_SEVERITY_DEFAULT"
      },
      {
        "Id": "TARGET_APPLY",
        "Severity": "LOGGER_SEVERITY_DEFAULT"
      },
      {
        "Id": "TASK_MANAGER",
        "Severity": "LOGGER_SEVERITY_DEFAULT"
      }
    ],
    "CloudWatchLogGroup": null,
    "CloudWatchLogStream": null
  },
  "ControlTablesSettings": {
    "historyTimeslotInMinutes": 5,
    "ControlSchema": "",
    "HistoryTimeslotInMinutes": 5,
    "HistoryTableEnabled": false,
    "SuspendedTablesTableEnabled": false,
    "StatusTableEnabled": false
  },
  "StreamBufferSettings": {
    "StreamBufferCount": 3,
    "StreamBufferSizeInMB": 8,
    "CtrlStreamBufferSizeInMB": 5
  },
  "ChangeProcessingDdlHandlingPolicy": {
    "HandleSourceTableDropped": true,
    "HandleSourceTableTruncated": true,
    "HandleSourceTableAltered": true
  },
  "ErrorBehavior": {
    "DataErrorPolicy": "LOG_ERROR",
    "DataTruncationErrorPolicy": "LOG_ERROR",
    "DataErrorEscalationPolicy": "SUSPEND_TABLE",
    "DataErrorEscalationCount": 0,
    "TableErrorPolicy": "SUSPEND_TABLE",
    "TableErrorEscalationPolicy": "STOP_TASK",
    "TableErrorEscalationCount": 0,
    "RecoverableErrorCount": -1,
    "RecoverableErrorInterval": 5,
    "RecoverableErrorThrottling": true,
    "RecoverableErrorThrottlingMax": 1800,
    "ApplyErrorDeletePolicy": "IGNORE_RECORD",
    "ApplyErrorInsertPolicy": "LOG_ERROR",
    "ApplyErrorUpdatePolicy": "LOG_ERROR",
    "ApplyErrorEscalationPolicy": "LOG_ERROR",
    "ApplyErrorEscalationCount": 0,
    "ApplyErrorFailOnTruncationDdl": false,
    "FullLoadIgnoreConflicts": true,
    "FailOnTransactionConsistencyBreached": false,
    "FailOnNoTablesCaptured": false
  },
  "ChangeProcessingTuning": {
    "BatchApplyPreserveTransaction": true,
    "BatchApplyTimeoutMin": 1,
    "BatchApplyTimeoutMax": 30,
    "BatchApplyMemoryLimit": 500,
    "BatchSplitSize": 0,
    "MinTransactionSize": 1000,
    "CommitTimeout": 1,
    "MemoryLimitTotal": 1024,
    "MemoryKeepTime": 60,
    "StatementCacheSize": 50
  },
  "ValidationSettings": {
    "EnableValidation": true,
    "ValidationMode": "ROW_LEVEL",
    "ThreadCount": 5,
    "PartitionSize": 10000,
    "FailureMaxCount": 10000,
    "RecordFailureDelayInMinutes": 5,
    "RecordSuspendDelayInMinutes": 30,
    "MaxKeyColumnSize": 8096,
    "TableFailureMaxCount": 1000,
    "ValidationOnly": false,
    "HandleCollationDiff": false,
    "RecordFailureDelayLimitInMinutes": 0
  },
  "PostProcessingRules": null
} Everything should be ok, right ?Let’s check for example some random table attributes on both databases: PROD-LIKE DB (the one we snapshotted): | portal_page | CREATE TABLE ( int(10) NOT NULL AUTO_INCREMENT, int(10) NOT NULL, int(10) NOT NULL, int(10) DEFAULT NULL, varchar(10) DEFAULT NULL, varchar(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL, enum('1','0') NOT NULL DEFAULT '1',   PRIMARY KEY ( ),   UNIQUE KEY ( , , ),   KEY ( ),   KEY ( ),   CONSTRAINT FOREIGN KEY ( ) REFERENCES ( ),   CONSTRAINT FOREIGN KEY ( ) REFERENCES ( ) ) ENGINE=InnoDB AUTO_INCREMENT=346 DEFAULT CHARSET=utf8 | portal_page id portal_id page_id section_id layout route_alias active id portal_page_section portal_id page_id section_id portal_id portal_id page_id page_id FKpp_page_id page_id default_pages id FKpp_portal_id portal_id portal id PROD-RESTORED, same table on the replicated RDS database: | portal_page | CREATE TABLE ( int(11) NOT NULL, int(11) NOT NULL, int(11) NOT NULL, int(11) DEFAULT NULL, varchar(10) DEFAULT NULL, varchar(255) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL, enum('1','0') NOT NULL,   PRIMARY KEY ( ) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 | portal_page id portal_id page_id section_id layout route_alias active id <a href="https://medium.com/media/7ffedd984d9bb9e803086cc8a94c5c80/href">https://medium.com/media/7ffedd984d9bb9e803086cc8a94c5c80/href</a> So we ended up with all the data in the database, no error show, but tables structure completely altered (or better, not replicated at all). Well, maybe it was my fault. Let’s open a support ticket to AWS. In a couple of days i received an answer (and trust me, Aws support guys are amazing and very technically skilled)…Basically, according to them: ”DMS takes a minimalist approach and creates only those objects required to efficiently migrate the data. In other words, AWS DMS creates tables, primary keys, and in some cases unique indexes, but doesn’t create any other objects that are not required to efficiently migrate the data from the source. For example, it doesn’t create secondary indexes, non primary key constraints, or data defaults”. Well, Houston we have a problem… We absolutely needed to have the identical data structure on the new database, so what now ?Since we are migrating from MariaDB to another MariaDB, AWS support guy suggest me to use a mysqldump to create an identical replica (…) But no, wait… we need live data replicated to avoid data losses.So, no old good mysqldump for us.But hey, wait a moment… it’s only a data structure issue, right ?So why not trying to dump ONLY the data structure from the old database, restore the data structure dump on the new database and then start again the replication task ? In this way we should be able to keep both structure and data consistency… ! (?) Only way to know, trying it.A good reference about mysqldump command can be found here: https://dev.mysql.com/doc/refman/8.0/en/mysqldump.html Once done, i started again my replication task with option enabled (this is the most important thing because it’s the only way to replicate your data and keep the data structure as it is. If you choose “Drop tables on target” for example, you’ll end up with the same situation we encountered on our first test, that means no data structure consistency during and after replication). “Truncate” And here we are, we now have an ongoing replication instance that will replicate every single change on the main database to the other one. Have fun! NOTE: All product names, logos, and brands are property of their respective owners.

What I learned by migrating an RDS database into production using AWS DMS

As described in git page: “Git is a free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.” — git-scm Let’s first analyse what does this means. Being a means that it allows you to create various versions of your project along the way. version control system, Being implies that it can be replicated along the network. It makes perfectly sense because it’s used by entire teams. distributed The first thing that it’s important to know is how git manages things to perform this . This article will focus in git foundations so that in next articles we can tackle more advanced concepts. version control How version control is made The first thing that must happen in order to use git, is to initialise a git repository inside a folder. Sure you can also clone an existing project, but let’s begin from the start. > mkdir git-project> cd git-project> git init Initialized empty Git repository in /Users/stupidgopher/git-project/.git/ So as you can see, an empty git repository was initialised in a .git folder inside our directory. In order to git save versions of your code you have to perform an action, make an interaction. So the first conclusion is that there is a point of interaction between you and git. Well this point of interaction is called commit. A commit can be described as a project change transaction to git. In order to commit some changes we must add files or folders to the staging area. The staging are can be described as a bucket where you put the changes you want to commit next. So let’s see this in action, by adding to stage our first file and folder: > mkdir folder> echo "a file" > folder/A> echo "a file" > B> git add folder B> git status On branch master No commits yet Changes to be committed:(use "git rm --cached <file>..." to unstage) new file:   Bnew file:   folder/A The first feedback line says that we are on master branch. Branches will be covered in the next article. For now think that master is our context. We added the folder and the file B and both of them are now changes to be committed. Let’s commit the changes, using the option to associate a message. Associating a message is mandatory, so if you don’t provide one with , the default system editor will prompt you one message. -m -m > git commit -m "Adding the first files" [master (root-commit) 967d0ac] Adding the first files2 files changed, 2 insertions(+)create mode 100644 Bcreate mode 100644 folder/A So we committed our first changes, creating the first version of our project. Let’s do the second commit so that you can start to see the big picture: > echo "one more line" >> B # >> appends a new line to the file> git add B> git commit -m "Add changes to file B" We have now two commits, the first one added the files to your git repository with the first lines. The second commit was built on top of the first one. Git is like a graph database, each commit is built on the top of the previous one, except for the first one that don’t have any ancestor. So from here you can already grasp a model like the one I am going to show in the next image: Graph showing a sequence of commits Every commit hides behind a snapshot corresponding to a version. This is an important feature of git in opposition to other version control systems that save the changes instead. Why this is important? Well if we are storing several versions of our project it’s expectable that in the future somehow we want to travel to a previous commit. By saving a snapshot instead of changes, we can go from point D to point A without revisiting intermediary points. Commits IDs and reference/pointers Like in any database it’s desirable that a record has an ID. Git is no exception, each time a commit is created a new unique ID is associated to that commit. In this case a is generated so that we can identify this commit later. sha1 hash Another important characteristic of a commit is that once is created cannot be changed. It’s immutable, so if we need to change anything including the message, a new commit must be generated. Let’s execute git log command with a couple of options that will make the output prettier: > git log --pretty --graph --oneline * d316650 (HEAD -> master) Add changes to file B* 967d0ac Adding the first files As you can see in this case our first commit has an hash starting with and the second commit . In case you are following along and reproducing my steps your hashes are going to be different. 967d0ac d316650 You may noticed that in the last commit we have (HEAD -> master). This is an indication that is pointing to and is pointing to the last commit. HEAD master branch master is a reference that points to the current state of your project. HEAD As you may have guessed the a branch is also a pointer, but with different characteristics from HEAD. This will be explained in the next article. Drill-down version storage To end this article in beauty we are going to drill-down the snapshot storage, using the commit as the start point. The commands that I am going to use are not essential to everyday usage. But in my opinion is important to know how things works under the hood to use git with more confidence. Until now a commit is defined by an and a commit . If we execute git log without options, we are going to see more information: SHA1 ID message > git logcommit d316650f83e2cc2ec2f44485c521b3f42872d403 (HEAD -> master)Author: *** < >Date:   Sun Jun 17 01:52:53 2018 +0100 ***@gmail.com Add changes to file B commit 967d0acee11b5156c48b833c24cc6696e6804bd3Author: *** < >Date:   Sun Jun 17 01:38:03 2018 +0100 ***@gmail.com Adding the first files Until now this is what we got: We know that a commit as a , and . We didn’t see how a commit references an ancestor and the snapshot of our tree at that version. message author date We are going to use a new command . Git cat-file provides content or type and size information for repository objects like described in . git cat-file git manual So git uses objects to store information. We are going to discover this objects in an interactive way, so let’s start: > git log --pretty --graph --oneline * d316650 (HEAD -> master) Add changes to file B* 967d0ac Adding the first files First let’s analyse the first commit: > git cat-file -t 967d0accommit # the type of this object is commit > git cat-file -p 967d0actree 1b8369392f39b30c2da28ae285476572394e145eauthor *** < > 1529195883 +0100committer *** <*** > 1529195883 +0100 ***@gmail.com @gmail.com Adding the first files So we executed first the cat-file with -t to see the type of commit object. As expected the type is . Secondly we executed cat-file with -p option “pretty print” and we have the author and message inside the object. But the new information is a tree reference. What is a tree object? commit > git cat-file -t 1b8369392f39b30c2da28ae285476572394e145etree > git cat-file -p 1b8369392f39b30c2da28ae285476572394e145e100644 blob 02f6335fc4f28cc4ea2d0846aacff267a149effb B040000 tree da97193026f17ed933494b723517904586648324 folder The tree object as his type says it’s a tree of files. In this case the tree object as two references. The first one is a and notice that in the front of this reference we have the filename in this case B. The second one is another , as expected, because as you know we added a file and a folder to our project. blob tree Let’s analyse the blob: > git cat-file -t 02f6335fc4f28cc4ea2d0846aacff267a149effbblob> git cat-file -p 02f6335fc4f28cc4ea2d0846aacff267a149effba file A blob it’s an object that stores the file content. Just for the sake of consistency let’s see the tree at the same level of this blob. > git cat-file -t da97193026f17ed933494b723517904586648324tree> git cat-file -p da97193026f17ed933494b723517904586648324100644 blob 02f6335fc4f28cc4ea2d0846aacff267a149effb A Wow, this tree as a reference to a file named with the same blob that we saw previously. This is because the content of file inside is the same as the content of file . Git is very clever, it only stores what he needs to store nothing more than that. A A folder B So we analysed the first commit that is a special case, because it hasn’t any ancestor. So let’s see the second commit: > git cat-file -t HEAD # HEAD > MASTER > current commit > git cat-file -p HEADtree 3c7e122c50226fdb638ad38f44beffe717d4922e author *** < > 1529196773 +0100committer *** < > 1529196773 +0100 parent 967d0acee11b5156c48b833c24cc6696e6804bd3 ***@gmail.com ***@gmail.com Add changes to file B Notice the reference in this commit to is ancestor. Finally let’s just see the tree. It’s important to have in mind that the second commit only introduces changes in file B. > git cat-file -p 3c7e122c50226fdb638ad38f44beffe717d4922e100644 blob 84a9ec3809e10f40ac7e1b248e036c0ce1237ee6 B 040000 tree da97193026f17ed933494b723517904586648324 folder The hash of the tree in folder remains the same, git only builds new blobs and trees for the changes. Everything that is untouched remains the same. how git keeps track of versions Hope you enjoyed this first article, sorry if it was too long. In the next article we are going to focus on branches. Peace, Stupid Gopher

Git from zero to hero - starting with foundations

To know an object is to lead to it through a context which the world provides. — William James (1842–1910) — ​ a temporal trace of operations — ​is vital for any mission-critical system, no matter how big or small. Same was the case with our , which is why we wanted to get it right, right from the beginning. Logging maintaining Project-X framework — where each log line automatically records its originating logical context, such as whether it came from a specific unit or from the base framework — ​was something we have been looking forward to, based on our experience with logging on the legendary . Contextual logging UltraESB We already knew , with almost everything that we need; but we needed more: log4j2 was offering contextual logging with its [CloseableThreadContext](https://logging.apache.org/log4j/2.x/manual/thread-context.html) implementation We needed a proper governance mechanism where each log line contains a unique log code, indicating the subsystem, module (package) and even an exact “index” of the specific log statement, so that we would no longer need to through the whole codebase to find out where the bugger came from. log code grep We needed to inject environmental variables and system properties, with a certain prefix, to be automatically injected to the logging context, so that specific applications could inject their runtime parameters to the logs (such as in case of our ). cluster ID Integration Platform We also needed to be API-independent of log4j2, as we should retain the freedom to detach from log4j2 and utilize a different logging framework (such as ) in case we need to. While we could have utilized a third-party wrapper such as we couldn’t find a wrapper that could readily fulfill all our demands. logback SLF4J Hence, as with the previous UltraESB, we wrapped log4j2 with , our own logging implementation. consists of an API and a set of bindings to real logging frameworks (like log4j2 and logback), one of which can be plugged in dynamically at server startup time with Java’s dear old mechanism. This helped us to avoid leaking of log4j2-specific stuff into our implementations, as the log4j2-based implementation (and hence log4j2 itself) could be completely removed from the set of compile-time dependencies. x-logging x-logging ServiceLoader from our team, who was also the originator of Project-X, hacked around with log4j2 for some time, and finally came up with a cool design to automatically propagate the current context of a log line, i.e. whether it originated from the platform (system, a.k.a. ) or from a deployed , and if it’s the latter, other metadata of the project (such as the version). The coolest part was that this context automatically gets cleaned up once execution leaves that particular context. Ruwan engine project If you are familiar with , this may all sound quite simple. For the rest of the crowd, it would be enough to mention that facilitates injecting key-value pairs to the logging context, such that when the context is closed, only the ones injected in the current context get cleaned up. The injected values are automatically made available to the logging context ( ) of the calling thread; or, in English, every log line printed by that thread sees the parameter in its thread context (or in old-school jargon). [CloseableThreadContext](https://logging.apache.org/log4j/2.0/log4j-api/apidocs/org/apache/logging/log4j/CloseableThreadContext.html) CloseableThreadContext ThreadContext MDC Okay, I admit the above might have been a bit hard to understand. Perhaps a sample snippet may do a better job: // assume we are walking in, with nothing useful in the context

try (CloseableThreadContext.Instance level1 =    CloseableThreadContext.put("level", "1")) {

    // now the context has "1" as "level"

    // will see {level=1} as // the context    logger.debug("Commencing operation");

    // let's also put in a "clearance" value    level1.put("clearance", "nypd");

    // now, any log lines would see {level=1,clearance=nypd}

    logger.debug("Commencing operation");    // let's also put in a "clearance" value    level1.put("clearance", "nypd");    // now, any log lines would see {level=1,clearance=nypd}

    // let's go deeper    try (CloseableThreadContext.Instance level2 =        CloseableThreadContext.put("level", "2")            .put("clearance", "fbi")) {

        /* now both of the above "level" and "clearance"           values are "masked" by the new ones           and yes, you can chain together context mutations */

        // will see {level=2,clearance=fbi}        logger.debug("Commencing investigation");

        // putting in some more        level2.put("access", "privileged");        // now context = {level=2,clearance=fbi,access=privileged}

        // still deeper...        try (CloseableThreadContext.Instance level3 =            CloseableThreadContext.put("level", "3")                .put("clearance", "cia")) {

            /* "level" and "clearance" are overridden, but               "access" remains unchanged */

            // {level=3,clearance=cia,access=privileged}            logger.debug("Commencing consipracy");

        }

        /* cool thing is, once you're out of the level3 block,           the context will be restored to that of level2           (thanks to the AutoCloseable nature of           CloseableThreadContext.Instance) */

        // {level=2,clearance=fbi,access=privileged}        logger.debug("Back to investigation");    }

    // same for exiting level 2

    // {level=1,clearance=nypd}; access is gone!    logger.debug("Back to operation");

}

logger.debug("Back to square one"); // {}; oh no, all gone! This mechanism was ideal for our use, as we needed to include the current execution context of a thread along with every log line generated by that thread: In , the underlying engine of , a maintained at the base framework level is responsible for processing inbound messages on behalf of an belonging to a particular project. Project-X UltraESB-X worker threadpool integration flow We consider the thread to be in the project’s context only after the message has been injected to the of a particular integration flow. The worker thread is supposed to do quite a bit of work before that, all of which would be considered to belong to a context. ingress connector system We generate logs throughout the whole process, so they should automatically get tagged with the appropriate context. Moreover, since we have specific error codes for each log line, we need to open up a new context each time we actually output a log line, which would contain the required log code in addition to the other context parameters. So, the life of a thread in the pool would be an endless loop of something like: // wake up from thread pool

// do system level stuff

loggerA.debug(143, "Now I'm doing this cool thing : {}", param);

try (CloseableThreadContext.Instance projectCtx =     CloseableThreadContext.put("project", project.getName())         .put("version", project.getVersion())) {

    // do project level stuff

    loggerM.debug(78, "About to get busy : {}", param);

    // more stuff, tra la la la}

// back to system level, do still more stuff Internally, , and others will ultimately invoke a method: loggerA loggerM logImpl(code, message, params) // context already has system/project info,// logger already has a pre-computed codePrefix

try (CloseableThreadContext.Instance logCtx =     CloseableThreadContext.put("logcode", codePrefix + code)) {

    // publish the actual log line}

// only "logcode" cleared from the context, others remain intact We simulated this behaviour without binding to log4j2, by introducing a interface: CloseableContext import java.io.Closeable;

public interface CloseableContext extends Closeable {

    CloseableContext append(final String key, final String value);

    void close();} whose log4j2 variant ( , obviously) will manipulate instances in the same manner: Log4j2CloseableContext CloseableThreadContext import org.adroitlogic.x.logging.CloseableContext;import org.apache.logging.log4j.CloseableThreadContext;

public class Log4j2CloseableContext implements CloseableContext {

    private final CloseableThreadContext.Instance ctx;

    /**     * Creates an instance wrapping a new default MDC instance     */    Log4j2CloseableContext() {        this.ctx = CloseableThreadContext.put("impl",            "project-x");    }

    /**     * Adds the provided key-value pair to the currently active     * log4j logging (thread) context     *     * @param key   the key to be inserted into the context     * @param value the value to be inserted, corresponding to     * {@code key}     * @return the current instance, wrapping the same logging      * context     */    @Override    public CloseableContext append(String key, String value) {        ctx.put(key, value);        return this;    }

    /**     * Closes the log4j logging context wrapped by the     * current instance     */    @Override    public void close() {        ctx.close();    }} Now, all we have to do is to open up an appropriate context via a nice management interface, : LogContextProvider // system context is active by default

...

try (CloseableContext projectCtx =     LogContextProvider.forProject(project.getName(),         project.getVersion())) {

    // now in project context}

// back to system context And in : logImpl try (CloseableContext logCtx =      LogContextProvider.overlayContext("logcode",         codePrefix + code)) {

    // call the underlying logging framework} Since we load the implementation together with the logger binding (via ), we know that will ultimately end up invoking the correct implementation. CloseableContext ServiceLoader LogContextProvider And that’s the story of contextual logging in our framework. x-logging Maybe we could also explain our log code governance approach in a future post; until then, happy logging!

Stories, News & Mentions about LOGCOMEX

Logcomex’s Stories on HackerNoon

Logcomex's News & Mentions