SPREAD makes product data accessible, intuitive, and actionable

SPREAD

spread

The decentralized finance (DeFi) ecosystem continues to expand across multiple blockchains, offering new opportunities but also introducing major challenges. One key issue is that of fragmented liquidity - assets and trading activity spread thinly across disconnected platforms. Another threat comes from weaknesses in cross-chain interoperability - bridges that promise to connect blockchains but with security flaws that leave funds at risk. To discuss these problems and potential solutions, I spoke with Boris Povar, a co-founder, and CEO of the project. Eywa Povar brings a strong technical background to tackle these complex topics. With a degree in mathematics and programming, he has worked in crypto since 2017 after a successful career running an IT outsourcing firm. As a portfolio investor and fund manager in the DeFi space, he has direct experience grappling with risks around fragmented capital and insecure bridges. Since 2020, he has focused his efforts on developing a novel architecture for trusted and efficient cross-chain communication. Povar provides insight into the scale and consequences of fragmented liquidity and flaws in today's bridges. The interview sheds light on the roadblocks hindering DeFi's progress as well as innovative solutions in development. Povar lends both technical expertise and real-world perspective to outline a vision for the next stage in DeFi evolution. What are the problems of fragmented liquidity and insecure cross-chain communication? The problem of fragmented liquidity can be formulated as follows: Liquidity located in one blockchain is isolated and not utilized in other blockchains simultaneously. Protocols addressing this issue enable your funds to operate across multiple blockchains/protocols concurrently. For instance, Liquid Staking protocols enable participation in the Ether consensus, earning income, while still utilizing funds in DeFi. Blast operates on a similar principle, where funds in their blockchain function within major DeFi protocols in Ethereum, generating income. However, examining this problem from the perspective of bridges reveals inefficiencies. Traditional lock-mint-burn-unlock bridges are ineffective with liquidity, as they connect assets and blockchains in pairs, requiring liquidity to be added to each link. Liquidity locked in such bridges remains dormant and is utilized ineffectively; L2 bridges serve as examples. Meanwhile, the TVL of bridges is rapidly growing and has reached the total TVL of DEXes (approximately $17 billion), necessitating more efficient utilization. Thus, it can be argued that bridges exacerbate the problem of liquidity fragmentation. The issue of insecure cross-chain communication is exemplified by the Multichain protocol. Projects like Curve and Fantom relied on it for cross-chain communications and liquidity transfer. However, Multichain malfunctioned at one point, resulting in losses for these projects. Their infrastructure was entirely dependent on Multichain, and finding and integrating alternative options took considerable time. Multichain, like many cross-chain protocols, had centralized elements, making its operation heavily reliant on teams. This scenario underscores the risks associated with relying on any single bridge as critical ecosystem infrastructure, which may lead to unforeseen "black swan" events. How big of an issue are these problems currently in DeFi and crypto overall? What are the consequences? High slippages in cross-chain swaps and complex UI/UX pose challenges for the majority of users. Projects are compelled to divide their liquidity across blockchains, incurring high maintenance costs. Furthermore, most bridges exhibit centralized traits. To date, bridge hacks have resulted in losses exceeding $2 billion. At EYWA, we endeavor to consolidate the liquidity of major blockchains, enhancing efficiency without compromising security and decentralization. What initially motivated you to tackle the problems of fragmented liquidity and insecure cross-chain bridges in the crypto industry? I've been actively addressing my own challenges. With involvement in asset management and DeFi capital placement since 2020, and in the industry since 2017, I possess an in-depth understanding of the problems and risks associated with cross-chain fund transfers. Drawing on my technical background, I've meticulously analyzed bridge hacks, resulting in the loss of over $2 billion. Consequently, I grasp what's necessary to ensure the security of cross-chain communications. Moreover, I recognize the imperative of holding liquidity long-term for bridge efficiency. I'm well aware of the inefficiencies in DeFi liquidity utilization, contributing to the dominance of CEXes over DEXes. The first wave of cross-chain bridges, while experimental, merely offered a pricier service without enhancing security or decentralization. I often phrase it as: "It’s still the same CEX, just in a new package." Hence, since late 2020, I've been dedicated to developing a genuinely decentralized cross-chain protocol capable of rivaling CEXes without compromising decentralization and security for development speed and convenience. How exactly does aggregated bridge and consensus architecture provide security for transfers? Most cross-chain bridges currently employ their own messaging protocols, which often fall short in terms of decentralization and entail various risks associated with teams, regulations, and management. It's fair to say that virtually all bridges on the market require trust, leaving the entire ecosystem vulnerable to potential "black swan" events. In response to these challenges, the Eywa team, in collaboration with the Curve team, is developing a trustless bridge that relies on a consensus of the most reliable messaging protocols available. We leverage Axelar, L0, Wormhole, Chainlink CCIP, and other messaging protocols to create a highly secure solution for messaging, custody, and liquidity transfer. By combining Eywa Token Bridge smart contracts with data validation consensus, we establish an Eywa Consensus Bridge that ensures security independently of any team and enables users to retain control over bridge liquidity. When transmitting information, it will be sent in parallel to several messaging protocols based on the user's choice, allowing users to specify preferred bridges and required quorums. The bridge's smart contract in the destination network aggregates transmission results and compares them until a quorum is reached. Users can customize the level of security needed, determining the number of checks. If any protocol returns incorrect data, the transmission will be halted. Imagine initiating a cross-chain transfer of a substantial sum. Initially, your funds are secured in the bridge's smart contract. Subsequently, a cross-chain transaction is initiated, transmitted, and verified concurrently by multiple data transfer protocols. In the destination network, a specialized smart contract receives and compares responses. If consistent information is received from all protocols, the final transaction executes your transfer. However, if any bridge is compromised, resulting in mismatched data, the transaction will be aborted, allowing you to reclaim your funds in the source network. The probability of all transfer protocols being compromised simultaneously approaches zero. How does utilizing Curve’s existing deep liquidity pools enable low-slippage cross-chain trading on CrossCurve? The more liquidity available, the lower the slippages, which is an evident concept. However, to elucidate how utilizes Curve pools, I must delve into its architecture. As mentioned earlier, the issue with bridge liquidity fragmentation lies in the fact that locked liquidity within bridges remains inactive in DeFi. CrossCurve To address this, we utilize Curve's LP tokens as the foundational liquidity. Leveraging Curve's ability to exchange LP tokens within the pool as standard tokens, we generate s-tokens - synthetic collateralized derivatives - by consolidating them on one public blockchain, known as Hubchain (currently Fantom). Subsequently, we establish stable liquidity pools on Curve by connecting s-tokens of similar nature. These pools offer minimal slippage. A typical exchange transaction traverses three blockchains, involving an imbalance operation to add liquidity to the Curve pool in the source network, followed by minting the corresponding s-token to the Hubchain, where it is exchanged for an s-token backed by the Curve LP token from the destination network. The s-token is then burned, releasing the LP token in the destination network, and the final step involves an imbalance withdrawal of liquidity from the Curve pool, transferring it to the user's wallet - adding liquidity, exchanging, and removing liquidity. The greater the liquidity in the pools we utilize, the lower the losses incurred. Additionally, this architecture enables more efficient capital utilization as the liquidity from all blockchains is consolidated in one location simultaneously, rather than in pairs. Consequently, CrossCurve requires substantially less liquidity compared to its competitors to achieve low slippage. What are the main benefits to projects and developers of listing their tokens on CrossCurve? Web3 projects can list their tokens on any chain against crvUSD/Stables/WBTC/WETH/Curve LPs and trade them simultaneously across all chains for any asset. They can also receive yield provided by Eywa, Curve, and Convex on provided liquidity, attracting new users from different blockchains. Fast scaling for the utility of your token in other networks, while also reducing the costs associated with maintaining cross-chain liquidity by keeping a single pool. How does CrossCurve simplify the user experience for cross-chain DeFi operations? We utilize cross-chain ZAP transactions, condensing a large number of transactions for the user. For instance, a typical exchange involves a series of transactions across three blockchains, yet the user only needs to initiate one transaction. How will the CrossCurve yield flywheel attract liquidity providers and projects to the platform? Since all CrossCurve pools are native Curve pools, we have access to various tools for managing incentives. These include direct incentives, veCRV bribes, Convex rewards, and other vote aggregators like StakeDAO, Lobby, and Yearn. Additionally, we can create our own votemarket, accumulate Curve and Convex votes, and participate in CRV Wars. Our plan involves using bonding mechanisms to attract the protocol’s own liquidity to the main pools. The first phase of our flywheel launch will focus on direct incentives for pools, veCRV bribes, and attracting our own liquidity through bonding. Subsequently, we will introduce a DAO and our own vote market. The issuance of the EYWA token will be exclusively available to long-term holders and members of the CrossCurve DAO, thereby strengthening the token's value. Curve liquidity providers will receive enhanced yields on their LPs, and we will provide them with a seamless method to contribute to CrossCurve pools. Projects will have the opportunity to offer incentives on their pools. However, to receive additional incentives from the CrossCurve DAO and partake in their distribution, they must become members.It's worth mentioning that we will assist Curve in distributing their stablecoin crvUSD, as it is their priority. In return, they will support us with incentives by voting for our main pools. How will CrossCurve boost adoption of less liquid layer 1s and layer 2s through liquidity bridges? We are currently constructing a cross-chain liquidity core by combining the largest blockchains by TVL. Subsequently, we will integrate emerging blockchains into our system and link them to this core, simplifying the process of liquidity migration between networks for liquidity providers. What role will the CrossCurve DAO play in solving liquidity fragmentation and boosting chain interoperability? CrossCurve will function as a transport hub for liquidity migration between different chains, offering additional income opportunities for liquidity providers. They will receive a base yield for providing liquidity on the original network, along with a yield for contributing their LPs to CrossCurve pools. Additionally, we intend to serve as a launchpad for Web3 projects seeking to establish multi-chain liquidity. This will enable them to utilize a single liquidity pool that serves all blockchains. How will votemarkets and bribing systems incentivize long term alignment between liquidity providers and the protocol? Long-term liquidity providers will become members of the CrossCurve DAO, engaging in the distribution of issuance among CrossCurve pools. They will also receive platform commission income and incentives from projects seeking augmented rewards. Where do you see the main competition in cross-chain DEX space, and how is CrossCurve positioned against them? My primary competitors include projects such as Stargate, Squid, Synapse, and Hashflow. We position ourselves as a solution primarily catering to liquidity providers and DeFi power users who necessitate seamless movement of substantial liquidity across blockchains. Additionally, we address the challenges faced by Web3 founders, enabling them to reduce the expenses associated with establishing and maintaining multi-chain liquidity while broadening their access to users across various blockchains, thereby diminishing their reliance on specific blockchain ecosystems. Our key advantages include the capacity to facilitate large-volume exchanges with minimal losses, akin to Curve's renowned capability, efficient utilization of liquidity, a seamless user experience, robust security measures safeguarding funds within our bridge, lucrative farming opportunities for liquidity providers, and an attractive proposition for projects seeking to establish their own liquidity pools. What chains and assets will CrossCurve focus on supporting in the initial launch stages? CrossCurve will focus on supporting the Ethereum, Arbitrum, Optimism, Polygon, Avalanche, BNB Chain, and Fantom chains along with the USDT, USDC, DAI, WBTC, ETH, crvUSD, 3crypto Curve liquidity pools, crvUSD-USDT Curve liquidity pools, and 3crv Curve liquidity pools assets. Do you intend to partner up with other major players, DAOs and bridges? If yes, in what capacity? To construct the Consensus Bridge, we are collaborating with Curve and will engage with prominent players in the cross-chain market — L0, Wormhole, Axelar, and Chainlink — as integrators of their solutions. For liquidity building, we have partnered with Curve and other entities within the Curve ecosystem, including Convex, StakeDAO, Yearn, and Lobby, to incentivize their users. We are also collaborating with yield aggregators like Beefy. We are currently in negotiations to integrate all new blockchains that Curve will support. Among those on the list are Mantle, Linea, Scroll, zksync, BNB Chain, and others. Additionally, we plan to identify a new Hubchain, a strategic partner blockchain. To enhance scalability and on-chain performance, we will collaborate with bridge aggregators such as Li.fi, Rubic, and Rango, as well as with wallets like Trust Wallet. In the future, we aim to extend support for liquidity aggregation from other DEXes, such as Balancer and Uniswap.

Fragmented No More: CEO of EYWA Explains Fixing Spread-Out Liquidity and Bridge Security

খণ্ডিত আর নয়: EYWA-এর CEO ব্যাখ্যা করেছেন স্প্রেড-আউট লিকুইডিটি এবং সেতু নিরাপত্তা

Schluss mit der Fragmentierung: CEO von EYWA erklärt die Behebung verteilter Liquidität und Überbrückungssicherheit

No más fragmentados: el director ejecutivo de EYWA explica cómo solucionar la liquidez dispersa y la seguridad del puente

Fini la fragmentation : le PDG d'EYWA explique la solution à la question de la liquidité étalée et de la sécurité du pont

अब खंडित नहीं: ईवाईडब्ल्यूए के सीईओ ने स्प्रेड-आउट तरलता और ब्रिज सुरक्षा को ठीक करने के बारे में बताया

断片化はもう終わり: EYWA の CEO がスプレッドアウト流動性の修正とブリッジセキュリティについて説明

더 이상 단편화되지 않음: EYWA CEO, 분산형 유동성 및 브리지 보안 수정에 대해 설명

Fragmented No More: CEO da EYWA explica como consertar a liquidez espalhada e a segurança da ponte

Больше нет фрагментации: генеральный директор EYWA объясняет, как исправить разброс ликвидности и безопасность моста

Artık Parçalanmış Değil: EYWA CEO'su Yayılmış Likidite ve Köprü Güvenliğinin Düzeltilmesini Açıklıyor

Không còn bị phân mảnh nữa: Giám đốc điều hành của EYWA giải thích cách khắc phục tính thanh khoản dàn trải và bảo mật cầu nối

不再支离破碎：EYWA 首席执行官解释解决流动性分散和桥梁安全问题

Spreadsheets have been a critical tool for managing information for individuals and organizations. However, manual spreadsheet tasks can be time-consuming and error-prone. This guide will show you how Python and Openpyxl, a powerful Python library for working with Excel files, can revolutionize how you manage your spreadsheets. We'll also explore the benefits of using for spreadsheet tasks and how Openpyxl can help you automate many tedious and error-prone spreadsheet tasks. Python In this article, we will learn how to perform simple data automation operations, such as Accessing Cell Values, Conditional Formatting, and working with Formulas and Calculations. You will be able to streamline your routine tasks with spreadsheets. Getting Started Let’s start with setting up your environment. Installing Python You can get your operating system's most recent version of Python from the . After that, simply adhere to the installation instructions to complete the process. official Python website Installing Openpyxl After that, add Openpyxl to your toolbox. Open your terminal or command prompt and use the following command to install Openpyxl using pip, Python's package manager. pip install openpyxl If your installation is successful, you will see the following output. Now you can use Openpyxl to automate your spreadsheet operations. Basic Spreadsheet Operations Imagine you have an Excel sheet filled with valuable data, such as employee information. For this tutorial, we will use the Excel sheet provided by . You can download it from the given link, and it will contain the following dataset. We will use this Excel sheet to show how to do various spreadsheet operations with Openpyxl. The Spreadsheet Guru Loading and Accessing Data Let’s start with accessing the Excel sheet we downloaded earlier to show how to load an Excel sheet on your local machine. Here is the folder structure of my project. Both the employee_data file and the file are in the same directory. working.py To load the Excel file with Openpyxl, execute the following Python code. import openpyxl

wb = openpyxl.load_workbook('employee_data.xlsx') It will not provide any output. But the code will be executed without any errors. Accessing Cell Values One of the easiest ways to prove that the above code opens the file is to access and display a cell value. Let's print the full name of the employee in cell B2. To access a certain cell value, you must first access the sheet it belongs to. In this case, the name of the sheet is Data. import openpyxl

#Load the Excel file
wb = openpyxl.load_workbook('employee_data.xlsx')

#Access the sheet
sheet = wb['Data']

#Access the cell value
first_name = sheet['B2'].value

#Print the cell value
print(f"The first name is: {first_name}") You will get the following output. This is a simple example of accessing cell values in an Excel sheet. Now, let’s do something more complex. Suppose you want to display all employees' full names and annual salaries stored in columns B and J, respectively. Python can help you achieve this task with the following code: import openpyxl

#Load the Excel file
wb = openpyxl.load_workbook('employee_data.xlsx')

#Access the sheet
sheet = wb['Data']

for row in sheet.iter_rows(min_row=2, values_only=True):
    full_name = f"{row[1]}"
    annual_salary = row[9]
    print(f"Employee: {full_name}, Annual Salary: {annual_salary}") Once you execute this code, you will get the following output. Let’s try another example. Filtering is one of the everyday operations you perform with Excel data. Let's filter all employees whose names start with 'E'. import openpyxl

wb = openpyxl.load_workbook('employee_data.xlsx')

sheet = wb['Data']

for row in sheet.iter_rows(min_row=2, values_only=True):
    full_name = f"{row[1]}"
    annual_salary = row[9]
   
    # Filter employees whose name starts with 'E'
    if full_name.startswith("E"):
        print(f"Employee: {full_name}") This code will give the following output. Modifying Cell Values Modifying cell values is a fundamental operation you will frequently perform in Excel. Here's an example to illustrate this. You must change an employee's phone number or address using a program. Let's say everyone previously based in Beijing is now relocating to Miami, and you want to modify the data accordingly and save it to a new Excel file. How can you make this change for multiple employees in one go? import openpyxl

wb = openpyxl.load_workbook('employee_data.xlsx')

sheet = wb['Data']

# Change cell values from Beijing to Miami
for row in sheet.iter_rows(min_row=2, min_col=13, max_col=13):
    for cell in row:
        if cell.value == 'Beijing':
            cell.value = 'Miami'

 # Save the modified workbook
wb.save('employee_data_updated.xlsx') In this code snippet, we created a new Excel file called employee_data_updated, where all the employees who lived in Beijing moved to Miami. Advanced Operations In this section, we will investigate more challenging operations with Openpyxl. Conditional Formatting Excel's conditional formatting function is very effective. Suppose you want to highlight in green all employees earning more than $50,000 annually. Follow these steps to set up the conditional formatting rules: import openpyxl
from openpyxl.styles import PatternFill

wb = openpyxl.load_workbook('employee_data.xlsx')

sheet = wb['Data']

# Define the green fill style
green_fill = PatternFill(start_color='00FF00', end_color='00FF00', fill_type='solid')

# Iterate through rows starting from the second row (assuming the headers are in the first row)
for row_number, row in enumerate(sheet.iter_rows(min_row=2, values_only=True), start=2):
    annual_salary = row[9]  
   
    # Check if the Annual Salary is greater than $50,000
    if annual_salary > 50000:
        # If the condition is met, apply a green fill to the Annual Salary cell
        cell = sheet.cell(row=row_number, column=10)  
        cell.fill = green_fill

wb.save('employee_data_updated.xlsx') You can easily understand the code by reading the comments. Once you run the code above, you will see the following change in your new Excel sheet file. However, if you want to make salaries greater than $50,000 bold and italic, you can use the following code: import openpyxl
from openpyxl.styles import Font

wb = openpyxl.load_workbook('employee_data.xlsx')

sheet = wb['Data']

# Define the bold and italic font styles combined
bold_italic_font = Font(bold=True, italic=True)

for row_number, row in enumerate(sheet.iter_rows(min_row=2, values_only=True), start=2):
    annual_salary = row[9]

    if annual_salary > 50000:
        # If the condition is met, apply the bold and italic font style to the Annual Salary cell
        cell = sheet.cell(row=row_number, column=10)
        cell.font = bold_italic_font

wb.save('employee_data_updated.xlsx') Here is the output of it. Formulas and Calculations Excel is well known for its formula capabilities. With Openpyxl, you can automate the application of these formulas. Let’s find the sum, average, and median of all salaries. You can find all salaries between the J2 cell and the J1001 cell. Although we can calculate these values using Python, we will use the Excel formula “=SUM(J2:J1001)” to show you how to automate Excel formulas. import openpyxl

wb = openpyxl.load_workbook('employee_data.xlsx')

sheet = wb['Data']

# Insert labels and formulas to find the sum, average, and median
sheet['O2'].value = "Sum"
sheet['P2'].value = "=SUM(J2:J1001)"

sheet['O3'].value = "Average"
sheet['P3'].value = "=AVERAGE(J2:J1001)"

sheet['O4'].value = "Median"
sheet['P4'].value = "=MEDIAN(J2:J1001)"

wb.save('employee_data_updated.xlsx') The updated Excel file displays equations on P2, P3, and P4 cells. Batch Processing of Multiple Files Consider a folder containing several Excel files, each requiring the same data processing steps. You can automate this batch processing with Python. import os
import openpyxl

# Define the directory containing Excel files
directory = 'excel_files/'

# Iterate through files in the directory
for filename in os.listdir(directory):
    if filename.endswith('.xlsx'):
        file_path = os.path.join(directory, filename)
       
        # Load and process each file
        wb = openpyxl.load_workbook(file_path)
        # Perform operations…
       
        wb.save(file_path) The processing of numerous Excel files is automated by this script, which helps you save time and effort. You need to update the directory paths according to your requirements and write the operations you need to perform. Combining Data from Multiple Sheets Another routine Excel task you will encounter is combining data from various sheets or workbooks. Python can make this process easier. For example, if you have sales figures for multiple quarters in separate workbooks, you can combine them using Python. import openpyxl

# Load the target workbook where data will be consolidated
target_wb = openpyxl.load_workbook('combined_sales.xlsx')
target_sheet = target_wb['CombinedData']

# List of source workbooks
source_files = ['sales_data_q1.xlsx', 'sales_data_q2.xlsx']

for source_file in source_files:
    source_wb = openpyxl.load_workbook(source_file)
    source_sheet = source_wb.active  # Assuming data is in the first sheet

    # Append rows from source to target, skipping the header
    for row in source_sheet.iter_rows(min_row=2, values_only=True):
        target_sheet.append(row)

# Save the combined data
target_wb.save('combined_sales.xlsx') With the help of this code, you can merge data from many sheets into one. You need to update the file names according to your needs. Best Practices Let’s see some of the best practices you can follow when automating Spreadsheet Operations with Python and Openpyxl. Optimizing Code Efficiency Efficiency is crucial, especially when working with enormous datasets. Use effective methods, eliminate unnecessary operations, and consider implementing multithreading or multiprocessing for parallel processing when optimizing your code. Version Control and Collaboration Using version control tools like allows you to keep track of changes, collaborate effectively, and keep a revision history for your automation scripts. Git Documentation and Comments With comments and docstrings, thoroughly document your code. It will help others to quickly understand your automation scripts and help you maintain your docs as they get bigger. Testing and Validation You must thoroughly test your automation scripts in a controlled environment before deploying them in production. Make sure they perform as expected and are capable of handling a variety of situations. You can learn about other third-party packages for working with spreadsheets using Python in this . article Conclusion Using Python and openpyxl to automate spreadsheets has many benefits. Python and Openpyxl offer the required tools and flexibility, whether creating reports, conducting data analysis, or speeding up data validation. You can gain fresh insights into data management and save time, money, and effort by becoming an expert in spreadsheet automation. We expect you to learn how to use openpyxl and Python to create innovative spreadsheets and to use automation as a fundamental skill in your daily work.

How to Automate Spreadsheets With Python and Openpyxl

: This article aims to clarify that the code provided is not a universal solution for all aspects of mathematical modeling. Instead, it serves as an educational tool, highlighting a specific step in the mathematical modeling process and employing Python for illustrative purposes, with the primary goal of enhancing awareness in this field. Disclaimer It is also important to remember that there are specific measures you must take when submitting/proposing a mathematical model project, such as one related to the field of biomathematics. These steps include, but are not limited to: defining the issue, selecting a model approach (two models will be discussed in this article), formulating equations, parameter estimation—often based on historical data, implementing numerical simulations by resolving differential equations analytically or using computational tools like and MATLAB, validations and sensitivity analysis, scenario testing, result interpretation, and so on. Python As I said earlier, this only highlights a specific step in the mathematical modeling process and employing Python for illustrative purposes. Content Overview Introduction The Purpose of Simulation The SIR Model What It Does / SIR Python Code Simulation Example What You Can Learn Understand Disease Spread Explore Parameter Sensitivity Evaluate Interventions The SEIR Model What It Does / SEIR Python Code Simulation Example What You Can Learn Model Latent Periods Evaluate Early Interventions Study Complex Outbreaks Conclusion 1. Introduction Understanding how infectious diseases spread is essential for maintaining the public's health because they have existed throughout human history for centuries. A potent tool for simulating and understanding the dynamics of disease spread is mathematical models. The (Susceptible-Infectious-Recovered) and (Susceptible-Exposed-Infectious-Recovered) models will be discussed in this article, along with how to simulate them using Python. SIR SEIR What Simulations Are For For public health professionals and epidemiologists, simulation is like a crystal ball. It enables us to foresee the potential spread of diseases under various circumstances and treatment options. These models aid in decision-making, effective resource allocation, and comprehension of the potential effects of various techniques. Let's examine two basic models namely; SIR Model, and SEIR Model. 2. The SIR Model What It Does / SIR Python Code Simulation Example The Susceptible, Infectious, and Recovered (SIR) model separates a population into these three categories. Based on variables like transmission rate ( ) and recovery rate ( ), it simulates how these compartments alter with time. β γ Before we exemplify a simulation using Python, it’s necessary to make “model assumptions” when working on mathematical models. In our assumption we’ll create model using LaTeX formular or code: \begin{align*}
\frac{dS}{dt} &= -\beta \cdot S \cdot I \\
\frac{dI}{dt} &= \beta \cdot S \cdot I - \gamma \cdot I \\
\frac{dR}{dt} &= \gamma \cdot I
\end{align*} Note: You can latexify this formular using Python. An example can be found on . https://github.com/google/latexify_py/blob/main/examples/examples.ipynb This article will not bother to write that Python code to convert LaTeX to a proper mathematical notation/equation, but I have used online editor such as so that you see the formula/equation assumptions glaringly below: https://latexeditor.lagrida.com/ : where S represents susceptible individuals, I represents infected individuals, R represents recovered individuals. The parameters β and γ govern the rates of transmission and recovery, respectively. The negative sign (i.e −β) indicates that the number of susceptible individuals (S) decreases over time. The dot notation indicates “multiplication.” In summary, these equations describe the dynamics of the SIR model, where the number of susceptible people decreases as they contract the disease (i.e. dS/dt), the number of infectious people rises as a result of new infections and falls as they recover (i.e. dI/dt), and the number of recovered people rises as the disease is treated (i.e. dR/dt). Because the change in each compartment depends on the multiplication of pertinent components, dots (.) is used to denote multiplication. Since all assumptions are set, we can then run a simulation using Python for  SIR Model and then visualize the dynamics: import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt

# SIR model equations
def SIR_model(y, t, beta, gamma):
    S, I, R = y
    dSdt = -beta * S * I
    dIdt = beta * S * I - gamma * I
    dRdt = gamma * I
    return [dSdt, dIdt, dRdt]


"""
Initial conditions (such as S0, I0, and R0) are not to be random but I hardcoded them with specific values. These choices are typically made based on the characteristics of the disease being modeled and the context of the simulation. Initial condition are set such that S0 = 99%, which indicates the proportion of susceptible individuals when the simulation starts. I0 is set to 1%, which indicates proportion of infected individuals to be 1% when the simulation starts. R0 is set to 0% which is expected that there are are no recovered individuals when the simulations start.
"""
S0 = 0.99  
I0 = 0.01  
R0 = 0.00  
y0 = [S0, I0, R0]

# Parameters
# β (beta) is transmission rate and I chose 30%. γ (gamma) is set to 1%
beta = 0.3  
gamma = 0.1  

# Time vector
t = np.linspace(0, 200, 200)  # Simulate for 200 days

# Solve the SIR model equations using odeint()
solution = odeint(SIR_model, y0, t, args=(beta, gamma))

# Extract results
S, I, R = solution.T

# Plot the results
plt.figure(figsize=(10, 6))
plt.plot(t, S, label='Susceptible')
plt.plot(t, I, label='Infected')
plt.plot(t, R, label='Recovered')
plt.xlabel('Time (days)')
plt.ylabel('Proportion of Population')
plt.title('SIR Model Simulation')
plt.legend()
plt.grid(True)
plt.show() For differences between and , I will like to point you to and for you to make sense of it further. scipy.integrate.ode scipy.integrate.odeint odeint ode What You Can Discover Running the SIR model in Python allows you to: : Observe the epidemic curve and visualize how a disease spreads through a community. Running Python SIR Model above, you can see the outcome/result in the graph below: Understand Disease Spread Try testing various beta and gamma values to observe how they affect the length and peak of the outbreak. Explore Parameter Sensitivity: By changing the parameters, you can simulate the consequences of interventions like Evaluate Interventions: social isolation or immunization. 3. The SEIR Model What It Does / SEIR Python Code Simulation Example By adding a "Exposed" compartment, the SEIR model expands upon the SIR model. It takes into account the period of incubation during which people have been exposed but are not yet contagious. How to emulate it in Python is shown here. import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt

# SEIR model equations
def SEIR_model(y, t, beta, sigma, gamma):
    S, E, I, R = y
    dSdt = -beta * S * I
    dEdt = beta * S * I - sigma * E
    dIdt = sigma * E - gamma * I
    dRdt = gamma * I
    return [dSdt, dEdt, dIdt, dRdt]

# Initial conditions
S0 = 0.99  
E0 = 0.01  
I0 = 0.00  
R0 = 0.00  
y0 = [S0, E0, I0, R0]

# Parameters
beta = 0.3    
sigma = 0.1   
gamma = 0.05 

# Time vector
t = np.linspace(0, 200, 200)  

# Solve the SEIR model equations
solution = odeint(SEIR_model, y0, t, args=(beta, sigma, gamma))

# Extract results
S, E, I, R = solution.T

# Plot the results
plt.figure(figsize=(10, 6))
plt.plot(t, S, label='Susceptible')
plt.plot(t, E, label='Exposed')
plt.plot(t, I, label='Infected')
plt.plot(t, R, label='Recovered')
plt.xlabel('Time (days)')
plt.ylabel('Proportion of Population')
plt.title('SEIR Model Simulation')
plt.legend()
plt.grid(True)
plt.show() The only difference in this case is the introduction of to explore different scenarios and understand the dynamics of infectious diseases with an “exposed” period. It demonstrates how it accounts for an incubation period before individuals become infectious. latent period rate (σ) What You Can Learn When you use Python to run the SEIR model, you can: Model Latent Periods Recognize the differences between the behaviors of immediately infectious illnesses and diseases with incubation periods. Evaluate Early Interventions Analyze the effects of isolation and early detection strategies. Study Complex Outbreaks For illnesses like COVID-19 where exposed people are a major factor in transmission, use SEIR. 4. Conclusion Python's simplicity and robust libraries, like SciPy, make it the perfect language for modeling diseases. And by performing these simulations with it, you're learning more about the dynamics of infectious diseases. This can equip you with verdicts and prowess that can help make decisions that are well-informed and improve your own ability to evaluate epidemics in the real world. There is much more to come after the SIR and SEIR models. There are other complex models such as SEIRS (Susceptible-Exposed-Infectious-Removed-Susceptible) Model, Spatial Models, Network Models, etc. Investigating intricate models, as well as real world data, such as geospatial data, epidemiological data, behavioural data, etc, and carefully examining the effects of interventions, such as vaccination strategies,  treatment availability, social distancing measures, you will develop your modeling abilities even further. Your knowledge of these modeling tools can significantly impact protecting public health in a future where understanding infectious disease transmission is vital. This is already too long, but I hope I have been able to show you how to simulate SIR and SEIR Models in mathematical model, using Python.

Simulating Infectious Disease Spread with Python: SIR and SEIR Models

পাইথনের সাহায্যে সংক্রামক রোগ ছড়ানোর অনুকরণ: SIR এবং SEIR মডেল

Simulation der Ausbreitung von Infektionskrankheiten mit Python: SIR- und SEIR-Modelle

Simulación de la propagación de enfermedades infecciosas con Python: modelos SIR y SEIR

Simuler la propagation de maladies infectieuses avec Python : modèles SIR et SEIR

पायथन के साथ फैलने वाले संक्रामक रोग का अनुकरण: एसआईआर और एसईआईआर मॉडल

Python を使用した感染症の蔓延のシミュレーション: SIR モデルと SEIR モデル

Python을 사용한 전염병 확산 시뮬레이션: SIR 및 SEIR 모델

Simulando a propagação de doenças infecciosas com Python: modelos SIR e SEIR

Моделирование распространения инфекционных заболеваний с помощью Python: модели SIR и SEIR

Python ile Bulaşıcı Hastalık Yayılımının Simülasyonu: SIR ve SEIR Modelleri

Mô phỏng sự lây lan của bệnh truyền nhiễm bằng Python: Mô hình SIR và SEIR

使用 Python 模拟传染病传播：SIR 和 SEIR 模型

Companies are generating an ocean full of data, with marketing teams swimming in customer insights and finance departments navigating through intricate transactions. And let’s not even talk about how enormous spreadsheet files generated by cybersecurity teams are. Unfortunately, spreadsheet platforms like Microsoft Excel and Google Sheets can’t handle big data spreadsheet files with millions or billions of rows. As of this writing, the . Whereas, Google Sheets has a . maximum row limit of Excel is 1,048,576 maximum cell limit of 10 million One of the best ways to smartly analyze big data spreadsheets without causing your browser to crash or your computer to run out of memory is using Gigasheet. Recently, I interviewed , the Co-founder of , a big data spreadsheet platform that allows its users to upload and analyze gigantic-sized spreadsheets packed with millions or billions or rows. Jason Hines Gigasheet What is your company in 2–5 words? The big data spreadsheet. Why is now the time for your company to exist? Spreadsheet platforms like Excel, Airtable, Smartsheet, and Google Sheets are not built for handling big data. These tools are incredibly powerful and offer tons of useful capabilities, but because of the limited capabilities that restrict users from working with huge datasets. That’s where Gigasheet is a game-changer. We have built this platform for users looking for something more powerful than Excel and Google Sheets. We want users to get insights from big data without having to write code or set up databases. You shouldn’t need a dedicated Data Science team or a Computer Science degree to get answers from really big datasets. And that’s what our sole focus is. Since spreadsheets are second-nature to most professionals, we didn’t want to reinvent the wheel with a new product with the same functionality as a sheet. Instead, we decided to simplify spreadsheets for huge datasets. With Gigasheet, we offer data science capabilities in a lite platform with features like data enrichment, grouping, filtering, and much more. In a line, we combine the power of big data with the convenience of spreadsheets to work seamlessly with large datasets. What do you love about your team, and why are you the ones to solve this problem? I worked with cybersecurity analysts and revenue teams for a long time in my career. I realized that the volume of the data they handled was so large and complicated that they had to go through too much hassle to export and import it into a database. Besides, not everyone had database skills. So, they had to wait for other teams to support them in the research process, creating lengthy dependencies. That was the core problem statement we wanted to solve. My first-hand experience of working with these analysts and observing their problems in real-time helped me build Gigasheet as a powerful solution for working with large datasets, minus the hassle. If you weren’t building your startup, what would you be doing? Wow, that’s a good question. To be honest, since I committed to building Gigasheet, I haven’t even considered doing anything else. If I wasn’t doing this, I guess I’d probably be building another company or helping someone else do it. At the moment, how do you measure success? What are your metrics? I believe the goal of a venture-backed startup like ours is to fund our growth. So, we’re measuring success based on the platform’s utility for our end-users and our ability to go forward with our product roadmap as planned. In a few sentences, what do you offer to whom? We offer a big data spreadsheet platform with several use cases and advanced capabilities. We serve users from multiple industries and roles, including eCommerce, cybersecurity, marketing, sales, analysts, and more. What’s most exciting about your traction to date? Just the sheer number of users and data we’re processing is kind of mindblowing. We recently processed our 5 trillionth data point and have more than 30,000 users on Gigasheet today. It’s wild to think about it, but we’re processing, on average, about 200,000 cells of data every second. Where do you think your growth will be next year? We've recently launched APIs to help users connect other systems to Gigasheet. This makes Gigasheet a powerful solution for enterprises, particularly heavy-duty applications like Snowflake or Salesforce. You can automate everything possible on the platform. So, whether you want to upload your data on a simpler interface, schedule import/export, or clean and inspect it, it's all possible with our enterprise API. We're also launching shared workspaces for teams. Multiple users from the same company use Gigasheet for collaboration on big files. That motivated us to expand beyond the single-user interface and add the shared workspace functionality to help teams work together seamlessly. Another cool thing on our radar is introducing more enrichments with our predefined partners. So, users can enrich massive data at scale without any coding skills. Tell us about your first paying customer and revenue expectations over the next year. Our first paying customer was an accountant in Canada. We never spoke to them, but it seems like they were successful. Each month we’ve been adding more customers, which is great, but our primary focus remains on building out more features in our product for 2023. What’s your biggest threat? As a startup, there are always 1,000 ways to die. The biggest threats are probably existential factors that are outside of our control - stability of financial markets and things like economic factors. As Paul Graham has said, “Startups rarely die in mid keystroke.” So, we keep building.

Beyond Excel and Google Sheets: Innovations in Handling Massive Spreadsheets Amidst the Big Data Rev

Hello! I wrote a blog on finding the a couple of months ago. Since then, I have found a better, faster, and easier method, which works in . last non-empty rows most cases Find the Last Row & Column You can copy and paste one of the following functions as per your needs. is meant for a bound script, is very easy to implement, and works on the currently active worksheet. FindLastRowNColBoundScript(), should be used with an independent script. The given code is very simple. You can modify this function and pass variables such as ID and Worksheet names as parameters to make this function more dynamic. FindLastRowNColIndependentScript() /**
 * Function should be used for bound script.
 * Function use getDataRange().getValues() method to 
 * find lastRow and LastColumn of that Spreadhseet.
 */
function findLastRowNColBoundScript() {
  const sheet = SpreadsheetApp.getActiveSheet();

  const data = sheet.getDataRange().getValues();
  const lastRow = data.length;
  const lastCol = data[0].length;

  console.log(lastRow);
  console.log(lastCol);

  return [lastRow, lastCol];
}

/**
 * Function should be used for independent script.
 * Function use getDataRange().getValues() method to 
 * find lastRow and LastColumn of that Spreadhseet
 */
function findLastRowNColIndependentScript() {
  const ss = SpreadsheetApp.openById("<Your Spreadsheet's ID>");
  const sheet = ss.getSheetByName("<Name of the worksheet>");

  const data = sheet.getDataRange().getValues();
  const lastRow = data.length;
  const lastCol = data[0].length;

  console.log(lastRow);
  console.log(lastCol);

  return [lastRow, lastCol];
}


/**
 * This script is for finding last non empty rows and columns in a spreadsheet. 
 * This works in most cases but not in some cases such as columns with checkboxes
 * It is created in such a way that it only works with bound script.
 *
 * Created by: Nibesh Khadka.
 * linkedin: https://www.linkedin.com/in/nibesh-khadka/
 * website: https://nibeshkhadka.com
 */ This strategy of using () to find the last row and columns works in most cases but few such as columns filled with checkboxes on their entire rows. DataRange Thank You for Your Time My name is Nibesh Khadka, and as a freelance automation expert, I specialize in automating Google products with Apps Script. If you need my services, let me . know Don’t forget to like and share this blog. Also published . here

How to Get the Last Non-empty Row of a Column of a Spreadsheet Accurately

Hey there, lovely ladies and fruity fellows! It’s your favorite Tutti Frutti Women talking to you, and we have a serious topic to discuss with you all. April is , and we need to spread the word about this sneaky disease that affects so many people every year. Esophageal Cancer Awareness Month Did you know that esophageal cancer is one of the fastest growing types of cancer in the world? It’s true! According to the American Cancer Society, there were over 19,000 new cases of esophageal cancer in the United States alone in 2020, and over 15,000 deaths from the disease. That’s a lot of people who need our help and support! occurs when abnormal cells develop in the lining of the esophagus, which is the tube that connects your throat to your stomach. There are two main types of esophageal cancer: squamous cell carcinoma, which occurs in the upper part of the esophagus, and adenocarcinoma, which usually develops in the lower part of the esophagus. Esophageal cancer for esophageal cancer include smoking, heavy alcohol consumption, being overweight, and having acid reflux. But don’t worry, there are plenty of ways to reduce your risk too! Eating a healthy diet full of fruits and veggies, staying active, and avoiding tobacco and excessive alcohol can all help keep your risk of esophageal cancer at bay. Risk factors Unfortunately, esophageal cancer often goes undetected until it reaches an advanced stage, which makes it more difficult to treat. That’s why it’s so important to you may be experiencing, like difficulty swallowing, chest pain, or unexpected weight loss. If you notice any of these symptoms, be sure to talk to your doctor right away. can make all the difference when it comes to beating this disease. pay attention to any symptoms Early detection options for esophageal cancer vary depending on the stage of the disease and other individual factors, but they may include surgery, chemotherapy, and radiation. Your doctor will work with you to determine the best course of action based on your individual case. Treatment But here’s the thing, ladies and gents — about esophageal cancer doesn’t have to be all doom and gloom. raising awareness That’s where comes in! We’re all about spreading the love and raising awareness in a fun and delicious way. Our NFT collection celebrates cancer warriors and survivors, and a portion of the proceeds goes toward cancer research and providing weekly health grants to members. Tutti Frutti Women By participating in our community and minting our NFTs, you can show your support for (esophageal) cancer awareness and help us make a difference in the fight against this disease. We’re a global community of warriors, survivors, and amazing people all united in the fight against cancer. So come join us, ladies and gents, and let’s spread the word about (esophageal) cancer in a way that’s just as sweet and vibrant as our tutti frutti colors! Remember, cancer awareness doesn’t have to be all doom and gloom. Let’s make it fun and creative! As part of our cancer awareness campaign, we invite you to create a visual that promotes awareness for esophageal, head and neck, or testicular cancer and tweet it using #TFWtalkscancer and tag @tfw_nft. Get your creative juices flowing and show us what you’ve got! Let’s work together to spread awareness and support cancer patients and survivors around the world! The best visual will win a #TFWnft as a token of our appreciation for raising awareness and making a difference. Sources: American Cancer Society. Key Statistics for Esophageal Cancer. Retrieved from https://www.cancer.org/cancer/esophagus-cancer/about/key-statistics.html Mayo Clinic. Esophageal Cancer. Retrieved from https://www.mayoclinic.org/diseases-conditions/esophageal-cancer/symptoms-causes/syc-20356084

Spreading the Word About Esophageal Cancer in a Delicious Way

Spread and rest operators are two of the most useful pieces of shorthand in JavaScript. Spread operators allow us to expand an array or object into its individual elements, while rest operators allow us to condense multiple elements into a single array or object. Spread operators are written using three consecutive dots (...), and they provide us with an easy way to break up an array or an object into its individual elements. For example, let’s say we have the following array: let array = [1, 2, 3, 4]; If we wanted to add a fifth element to our array, we could use the Spread Operator like this: let newArray = [...array, 5]; The Spread Operator will take our array, expand it into its individual elements (1, 2, 3, 4), and then add the fifth element (5). The result is a new array that is one element longer than the original: newArray = [1, 2, 3, 4, 5]; The rest operator is essentially the opposite of the spread operator. It collects multiple elements into an array. This is useful if you don’t know how many arguments a function may receive, and you want to capture all of them as an array. The spread and rest operators are useful for quickly and easily manipulating collections and arguments. They make code much cleaner and easier to read, as well as reduce the number of lines of code needed. They’re extremely versatile tools, and every JavaScript programmer should know how to use them.

 JavaScript Spread and Rest Operators: A Guide

The Javascript spread operator permits an array or string, or other iterable, to be expanded in places with zero or more arguments (for function calls) or elements. The syntax for this is three periods (…) In this article, I will explain the use cases of the spread operator in JavaScript, so let’s jump right to it. Use Cases Adding the elements of an existing array into a new array let newarr = ["Red", "Yellow"];
let otherarr = ["Blue", ...newarr, "Green"] 
console.log(otherarr); // ["Blue", "Red", "Yellow", "Green"]; So let’s explain what is happening here, is an array containing two strings and , then the second array otherarr contains two strings and the . Usually, when we add this array without the three periods operator it will add the new array into the other array. But as we have the spread operator it will add the element of the array into the other array. newarr Red Yellow newarr You can pass elements of an array as arguments to a function function addThreeNumbers(x,y,z){
   return x+y+z;
}
let args = [1,2,3];
addThreeNumbers(...args); // 6 Another use case is that you can pass an element of an array as a function. in the above example, we have a function with three parameters which return the sum of these three numbers. Now we have an array with three values 1,2,3. In a normal state, we can pass three arguments to the function but with the help of the spread operator, we just pass the array with the three-dot operator (…), which will spread out the array to each element of the function. addThreeNumbers x,y,z Note: If we have a fourth element in the array it going to ignore the fourth element in the array, like the example below. function addThreeNumbers(x,y,z){
   return x+y+z;
}
let args = [1,2,3,4];
addThreeNumbers(...args); // 6 . Copy arrays let arr1 = [1,2,3];
let arr2 = [...arr1] If we have arr1 we can use the spread operator to copy the array into the second array. Concatenate array let arr1 = [1,2,3];
let arr2 = [5,6,7];
arr1.concat(arr2); // [1,2,3,5,6,7]
arr1 = [...arr1 ...arr2]; // [1,2,3,5,6,7] Another use case is that we can concatenate arrays together, normally how we do this, is we use the method to concatenate arrays. With the spread operator, we can concatenate arrays too. concat Reference Also published here. Lead image generated with stable diffusion.

Exploring Common Uses Cases of the JavaScript Spread Syntax (…) 

In recent years, developers have created sophisticated tools to make the job of analyzing big data easier. Popular open-source tools for Python include Pandas, NumPy and of course, there are math-oriented applications like Matlab and R, as well as SQL for databases and cloud-based data lakes.  Big Data solutions are becoming increasingly complex as data teams get more sophisticated, but this is leaving millions of part-time analysts overwhelmed. The powerful tools I mentioned above (and countless others) allow users to perform various data analysis operations, but these applications require a high level of technical skill and training to accomplish even the most basic tasks. Often the stakeholders, in the business context, don’t have the skills needed to analyze the data themselves. These users typically lean on an intermediary data team, bogging them down with the most banal of tasks. But what do you do if you don’t have a data team? It’s no wonder newcomers to the big data world struggle. Without prior coding or database experience, many find these highly technical tools overwhelming. Spreadsheets are widely used by business users, but Excel’s max row limit and reliance on loading the full dataset into the machine’s memory inhibits working on projects involving data analysis at scale. So, what’s a business analyst to do when working with large volumes of data? I hear the detractors muttering “if you're working with more data than Excel can handle, you should use a database.” To which I respond by reminding them that relatively few people in the world know how to use SQL (maybe 3 million), and there are 750 million users. Excel Enter Gigasheet Our no-code, big data spreadsheet, that can be used for analyzing datasets that typically require extensive IT infrastructure and data teams. Even at the (free) Community level, Gigasheet makes it easy to explore and analyze big data, as well as identify trends and anomalies. In this article, I will walk through 4 common big data transformations, and show you how anyone with basic spreadsheet skills can do them with just a few clicks using Gigasheet. Big Data Exploration In A Spreadsheet In some cases, data sets can span multiple gigabytes and even terabytes. Exploring these data volumes requires powerful systems, efficient methods of data storage and retrieval, and advanced techniques to analyze the data. Commonly used approaches include file replication and splitting, data sharding, and distributed computing. But what happens when you want to explore big data without all of this technological firepower? What if you’re not even sure what data a file contains? If only there were any easy way to visualize multi-gigabyte data files online, where complexity could be hidden from view, and the power and scale of the cloud could be leveraged. Fear not, one of Gigasheet’s many use cases is as a . Data not in CSV format? Not to worry - the system converts most structured data files on the fly. Simply upload your file and you’re on your way. free online CSV file viewer Combining Multiple Large Data Files Large data files are often split into multiple parts to make them easier to store, transfer, and process. Splitting a large file into smaller parts also reduces the risk of data corruption and makes it easier to recover lost data. However, when it comes time to analyze the data it’s important to have a comprehensive view, so these pieces must be merged, appended, or otherwise combined. The process of combining data from multiple sources into a single dataset can be done through process automation, data integration tools, or machine learning algorithms. While these methods are very powerful and capable, they are out of reach for the average business user. Gigasheet makes it simple to or Excel workbooks to JSON. To do this, simply upload the files as a Zip. Once decompressed, just select two or more files in your library. Then, use the Combine button in the Library to merge the files of the same structure. join multiple files together, from CSVs For instance, if you have 28 daily logs from the same server, you can easily merge them into one sheet using the Combine feature. Removing Duplicate Data Cleaning big data files of duplicates, aka de-duping, can be tricky, especially when you want to check for duplicates across multiple fields. Many users are familiar with techniques to , but few could tackle the task in SQL or Python. remove duplicate rows in excel based on two columns Removing duplicates based on multiple values is easy in Gigasheet, and works similarly to popular spreadsheets. Unlike the typical spreadsheet, Gigasheet scales to billions of records. Once data is loaded into Gigasheet, you’ll find a variety of Data Cleanup tools including a Delete Duplicates function. Simply select multiple columns when running Delete Duplicates and the cloud application will take care of the rest. Extracting Structured Data From JSON JSON (JavaScript Object Notation) is a popular data format for exchanging data between systems, applications, and services. It allows for storing and querying data in a structured and efficient manner. This is why most programming languages support reading and writing JSON data, and many APIs use JSON data. However, if spreadsheets are your go-to analysis tool, analyzing large datasets with JSON records can be tricky. You can of course open moderately sized JSON files in tools like , but if you’re working with highly nested JSON structures that are multiple Gigabytes in size, you’ll need to use a database…until now. Notepad++ Gigasheet converts, or “flattens,” huge JSON files on the fly, and they can easily be pared down, exported to CSV, and opened in typical spreadsheet software. Gigasheet accepts two possible JSON file structures: either an entire file as a JSON object, or JSON where there is one object per line. In the case of the latter, each JSON object becomes a row. Gigasheet handles each JSON file’s unique structure by creating a column for each value for the various nested and sub-nested objects. This results in a tabular representation of the repeated key data. Common fields have values represented across rows in the same column, and unique fields show values in their own column. It’s quite possibly the easiest way to convert JSON to a CSV. Wrapping Things Up We all know big data analysis is an essential part of modern businesses. I hope this article has presented some of the most commonly used solutions and techniques for exploring, combining, and analyzing mega-sized datasets with a free no-code alternative.

4 Data Transformations Made Spreadsheet-Easy

4 transformaciones de datos hechas con hoja de cálculo fácil

4 transformations de données simplifiées pour les feuilles de calcul

4 डेटा रूपांतरणों ने स्प्रेडशीट को आसान बना दिया

スプレッドシートで簡単にできる 4 つのデータ変換

4 transformações de dados facilitadas em planilhas

4 chuyển đổi dữ liệu được thực hiện dễ dàng trên bảng tính

4 数据转换使电子表格变得简单

THE RISE AND SPREAD OF BUDDHISM

After the meteoric rise of FTX as a crypto exchange, its sudden collapse has sent shockwaves through the industry. Sam Bankman-Fried, who had positioned himself as the poster boy of crypto, has lost . almost everything But that’s not the end of it. The contagion effect could spread much wider than anticipated over the coming weeks. The FTX fiasco is going to have a dramatic impact not just on centralized exchanges but also on blockchains and the broader DeFi and Web3 ecosystems. How? Allow me to explain. The Visible Cracks in Decentralized and Multi-Chain Web3 For a long-time, Ethereum was the go-to blockchain for developers to build their dApps. It offers a comprehensive developer toolkit and interface that helps to streamline production. What would take a developer months on another network could only take a few weeks on Ethereum. But its slow transaction speeds and sky-high gas fees prompted developers to look for alternative blockchains that promise to overcome Ethereum’s limitations. That’s what sparked the rise of the so-called “Ethereum-killer” chains like Solana, Cardano, and others. The future certainly looks to be , consisting of a variety of Layer-1 and Layer-2 chains, each vying to overcome central problems and mark itself as a market leader. multi-chain Developers are increasingly building dApps that can tap into the user bases and assets of multiple chains rather than being locked into just one ecosystem. As the FTX debacle has exposed, even investors betting on the multi-chain future are left scratching their heads. Why? They have no clue which blockchains would survive, let alone dominate, the multi-chain era. Solana blockchain, one of the most hyped “Ethereum killers,” is experiencing , thanks to Sam Bankman-Fried for revealing that it might not be as “decentralized” as many assumed. major stress Solana claimed to be fairly decentralized, meaning no single individual or entity could control or influence it. However, FTX had built a DEX called Serum on the Solana blockchain. Even though Serum claimed to be operated by a DAO, Alameda Research could make decisions about Serum without involving any other DAO member. Talk about decentralization! Besides building a DEX on Solana, SBF’s Alameda Research participated in Solana’s $314 million funding round last year. According to , the Solana Foundation had tens of millions of dollars in crypto assets stranded on the pre-collapse FTX. Coindesk The foundation also held 3.24 million common stock shares in the bankrupt exchange. Core Principles and Infrastructure Will Take the Industry Forward These revelations don’t mean Web3 is dead. Far from it. Sure, in this brutal bear market, many more centralized exchanges could bleed. Some of the revolutionary blockchains could turn out to be, well, not-so-revolutionary. But the foundational principles of blockchain - decentralization, trustlessness, and transparency - will continue to drive the industry forward. And there will be enough room for multiple blockchains to co-exist because each chain has certain unique aspects that make it attractive to users. The ability to leverage them for distinct attributes will allow users and developers to get the very best from the blockchain ecosystem. The pick-and-shovel providers, especially the decentralized ones, are also well-positioned to survive and thrive as the multi-chain ecosystem continues to take shape. That’s because they provide the underlying infrastructure to dozens of chains, and irrespective of which chains come to dominate the future, they will do just fine as long as the overall industry grows. For instance, Ankr is a leading Web3 infrastructure provider serving over 8 billion daily RPC requests across 39+ supported blockchains. Recently, Ankr upgraded its entire network to transition from a centralized platform to a fully decentralized protocol. It enables developers to build safe, scalable, and upgradable dApps on top of the supported chains. Ankr is like a messenger that relays on-chain information between blockchain nodes, dApps, and end-users so they can execute necessary tasks like transactions, populate wallet balances, fetch ownership information, and more. Ankr will remain useful as long as developers continue to build decentralized applications and services. And as the saying goes, the bear market belongs to builders. Final Thoughts No doubt it’s a difficult time for the industry, but difficult times are known to push developers to focus on strengths, build useful products and services for the future, and propel the industry forward. The blockchain ecosystem will continue to expand rapidly, become more interconnected, and attract more users. The events that have transpired over the last week are dramatic, but projects that uphold the original principles of blockchain and cryptocurrency will stand steady.

The FTX Contagion Could Spread Far and Wide, but the Web3 Backbone Unlikely to Be Affected

The , or (...), is a type of syntax in Javascript that is used by both function calls and arrays/objects. It has a multitude of different uses, so let's take a look at how we use the spread syntax in real Javascript code. spread operator spread syntax 3 dots In function calls We can use the 3 dots in Javascript function calls to convert an array into a set of arguments for a function. Let's look at an example. Below, our array is converted into the values for , , , and . x y z a let numbers = [ 1, 2, 3, 4 ];

let myFunction = function(x, y, z, a) {
    return x + y + z + a;
}

// Returns 10
myFunction(...numbers); , so the following is also valid, using the same function as before: This can be combined with other values let numbers = [ 1, 2 ];

// Returns 15 (i.e. 5 + 7 + 1 + 2)
myFunction(5, 7, ...numbers); This can also be used when calling a constructor with , for example: new let numbers = [ 1999, 26, 3 ];

let thisDate = new Date(...number); Merging Arrays Another useful way to use the spread syntax is to merge arrays. For example, we can merge two separate arrays into a new one using two spread syntaxes: let x = [ 1, 2, 3 ];
let y = [ 4, 5, 6 ];

// Returns [ 1, 2, 3, 4, 5, 6 ]
let newArray = [ ...x, ...y ]; Similar to before, we can combine this with other values and still get the same outcome: let x = [ 1, 2 ];

// Returns [] 4, 5, 1, 2 ]
let newArray = [ 4, 5, ...x ]; Merge Objects Finally, we can use the spread syntax to . In the below example, we merge two objects with key/value pairs into one object: merge objects let obj1 = { name: "John" };
let obj2 = { age: 114 };

// Returns { name: "John", age: 114 }
let newObj = { ...obj1, ...obj2 }; , the second object will take precedence and overwrite the first one, as shown below: If we try to merge two objects and there is a duplicate key let obj1 = { name: "John" };
let obj2 = { name: "Jake" };

// Returns { name: "Jake" }
let newObj = { ...obj1, ...obj2 }; - they let us run functions with arrays easily, and are good for merging objects and arrays. And that's how spread syntax work Also published . here

Decoding the Three Dots (…) Or Spread Operator in Javascript

I have some terrible news: If you are a human resources manager, your job is obsolete — you are no longer needed. It is only a matter of time before you are shown the door. Automation in human resources — the process of enhancing the efficiency of the HR departments by freeing employees from tedious manual tasks — is already here. The shock doctrine Human resources management — or rather, human-managed hiring and onboarding — is dead. I don’t know the exact date; but what I am sure about is that one day they dragged it out of its cubicle; took it to the back of the building, near the loading docks, and put a metaphorical bullet in its skull. They then dumped its body in a recycling bin, along with decommissioned 80386-based PCs, daisy-wheel printers and boxes of un-opened “ phones. After years of faithful service, it met its end by being unceremoniously dumped like yesterday’s coffee. DynaTAC 8000X mobile” My apologies for my bedside manners. I exaggerate for effect; but Automation in human resources — the process of enhancing the efficiency of the HR department by freeing its employees from tedious manual tasks — is already here. “Kitty couldn’t fall asleep for a long time. Her nerves were strained as two tight strings, and even a glass of hot wine, that Vronsky made her drink, did not help her. Lying in bed she kept going over and over that monstrous scene at the meadow.” , ² ³ How it all began If you are a human resources manager, your job is obsolete — you are no longer needed. It is only a matter of time before you are shown the door, that is, if you have not been fired already. Do not be shocked. This was a long time coming; you saw the first signs long ago — you just refused to believe you were next. A recent study on the changing roles of the Human Resources professional found that 50% of respondents believed that traditional Human Resources functions would, Allow me to explain: ¹ “…continue to grow into a more strategic function as administrative responsibilities are automated or outsourced to others…” In early 2008, newspapers around published a story that created a minor buzz. It claimed that a Russian publishing house — Astrel SPb — was releasing a book completely written by a computer. The book was to be called True Love and was a variation of the classic novel Anna Karenina written in the style of Haruki Murakami . It was widely reported the publisher claimed a group of Russian developers and philologists collaborated to create a computer program that generated the manuscript. ⁴ ⁵ In early 2016, Nishad Nawaz, a researcher at The Kingdom University in Bahrain published a paper titled, . In the abstract, Mr. Nawaz states, ⁶ Automation of the HR functions enhance the professional efficiency of the HR Professionals-A Review “…In many organizations, the human resource department is responsible for many strategic tasks from managing the hiring to [the] termination of employee[s], for example monitoring of employees’ at all the levels, handling payroll, managing employee[s'] benefits and so on. To make this work easier[,] organizations across the world are investing in HR automation [to] [carry] out the best human capital decision[s]…” I know what you’re thinking: — but you would be wrong to think this way; because the change is not only already happening, but it is accelerating. “…my company’s board of directors is too visually impaired to consider what kind of impact these new-flanged capabilities will have on the company to actually consider them let alone implement them…” While it is true that some companies have not fully considered implementing a complete, top-to-bottom HR automation strategy — largely because such a thing is still too abstract a problem and a not-so-clear-opportunity right now — news like Amazon’s drive to will not stay secret for long. automate hiring and onboarding for its hourly warehouse workers Do not kid yourselves, while corporate boards are not known for being bastions of innovation and forward-leaning thought, it’s possible — even if they are unable to see its affect on the corporation’s current business — at least, not yet, anyway. they know In an interview for the Guardian newspaper discussing his new book, ; Daniel Susskind, an economist and lecturer at Balliol College, Oxford is quoted as saying, ⁷ A World Without Work: Technology, Automation, and How We Should Respond “…When professional work is broken down into its constituent tasks a great many of these turn out not to be so complex after all, and can be done by less qualified people with the support of systems, or indeed by machines…” , an independent UK Bank — not affiliated with any other banks or organizations — had begun implementing an HR automation strategy to — as they put it, Metro Bank PLC “…automate processes so that employees can spend more time on customer interaction; for those customers who still want this person-to-person interactions…” The problem with this, as Danielle Harmer, chief people officer at Metro Bank puts it, “…is that automation could happen for the wrong reasons — namely cost-cutting instead of enhancing what the business does…” Remember this words, because you will hear this line many times from this point onwards. cost-cutting instead of enhancing what the business does Sooner or later — if it is not happening already — you will see a memo from upper-management announcing the introduction of automated applicant tracking software, automated hiring and recruitment tools such as Mya , Smashfly , BreezyHR and others as a,  “ namely focusing on complex tasks like decision making and strategizing. ⁸ ⁹ ¹⁰ cost-cutting measure [to] [enhance] what the HR does best”, Remember: These tools use AI to . And if that alone doesn’t give you pause, remember the quote above: significantly reduce the time to hire and accelerate the screening process “…When professional work is broken down into its constituent tasks a great many of these turn out to be NOT so complex after all, and can be done by less qualified people with the support of systems, or indeed by machines…” The bottom Line: Be afraid…be very afraid Corporations are profit-making enterprises. Their impetus and is to make money — or as the prospectus puts it When presented with an opportunity to reduce costs, increase efficiency, and enhance shareholder's value; they — corporations behaving as people, for the purpose of tax laws — have an almost biological tendency to seize it. raison d’être “..to deliver value to the shareholders…”. “It’s all ‘bout the money It’s all ‘bout the dun dun do do do dumb I don’t think it’s funny to see us fade away It’s all ‘bout the money” Let’s be honest, what human-centric Human Resources Management urgently needs is a clue about how technology is, and will continue to — perhaps more radically and suddenly than ever before — shape the future of hiring, onboarding, management and termination. Sadly, human resources managers have been woefully unprepared to cope with this new reality. Years of and a tendency for atrophying complacency have opened the door for its own destruction. hiring-by-spreadsheet Yes, we are in the early stages; but technology-driven change moves at the literal speed of light. Shipping and logistics leaders like Amazon — because of their high tech pedigree — are the first out the door proclaiming their embrace for HR automation, or HR Management by Algorithm. But make no mistake, others (domino’s, Jet, Casio, Comcast and more) are already following , in their footsteps. What’s worse, Cloud HR companies that offer HR support tools to enhance human-centric HR Management will expand, or are already expanding, their offerings the take advantage of advancement in Machine-Learning technologies applied to HR management. ¹¹ ¹² The promise of lower HR costs, more efficiency, in the hiring process, and other — more  tangible — benefits will prove irresistible to corporations large and small. The fact these measures will free more funds to compensate upper management in the form of performance bonuses and other perks, is just a happy coincidence. less human bias Try to understand this The machines — or rather the algorithms — that will replace you and take over your job are already here. They are plugged into your companies’ systems and have been learning from you all this time — they are faster, cheaper and more efficient. The process that will culminate in your obsolescence and termination has been under way for quite some time; you just haven’t noticed the metaphorical bullet hole in your skull and the shape of the recycling bin — but yeah, as an human-biased HR professional, most likely than not, you’re, metaphorically-speaking already dead. Author's note: I can't believe I have to say this, but If you are an HR Professional reading this article after googling my name, because I have applied for a position at your company, PLEASE DO NOT KILL THE MESSENGER. I am just reporting the facts. Be angry at your company's upper management. They are the ones plotting against you. Follow me (and I will follow you) on | | | | Twitter Medium Facebook Instagram YouTube

HR by Spreadsheet vs. HR by Algorithm

TLDR; An in-house CSV/Spreadsheet import tool is not an easy thing to build, and the number of companies that do it well are few and far between. Inefficiencies in data imports are often mitigated by troubleshooting articles or FAQs or handled by support calls. Poor error messages, helpdesk articles and FAQs put the onus on the customer to figure out what went wrong where. Support calls take up time and effort (and as a result, money) to resolve. Your customers are as busy as you and your developers are, and do not have the time to troubleshoot a less-than-user-friendly import experience. Your data importer is often the first experience the customer has with your products and services and a challenge here can cost you many opportunities. A strong CSV importer can be the difference between onboarding and attrition. Data is the lifeblood of your business, and obtaining it quickly and easily is the goal of any business application. In order to grow as a business, your method of data import needs to be as seamless as possible, allowing your customers to provide you the information you need without delay. A CSV importer is often the solution, sparing customers the pain of manually copying data into your database. Unfortunately, this means you need to develop/build a way to integrate this customer data in as efficient a manner as possible. Or do you? Developing a CSV importer is a difficult task, one that requires significant investment development time. Best case scenario, it is equivalent to building a feature of your application/site/solution. Worst case scenario, it is like building a separate product altogether. And this investment of time and energy is not guaranteed to be successful. That’s where a pre-built tool can become a life-saver, particularly if it can be integrated into your existing framework. is just what the doctor ordered...a white-labeled, customizable interface solution that can handle complex data imports, and which integrates seamlessly with your web page or application. Flatirons Fuse What A Poor Import Experience Looks Like Building a CSV importer is no easy task, and there is nothing worse than seeing customers struggle with a solution you have invested time and effort into.  We have seen numerous issues derail the customer’s CSV import experience, including, but not limited to the issues below: Unclear Guidance External data is not often going to be naturally configured to the requirements of your system. Some level of data manipulation is typically required. But if it is not clear what changes are needed, the import is all but guaranteed to fail. There are so many problems that can prevent an import from succeeding. Clear guidance can solve most of them. For example: What fields are required? What types of data need to go into those fields? How do I get all my existing data into a new template? Is there a size limit to what the importer can handle? Does the importer only accept a particular CSV format? (UTF-8, for example) Does the importer accept spreadsheet formats such as XLS or XLSX? Do columns need to match? In a particular order as well as name? How do I fix my data? For example, does location data need to be input as GPS coordinates, or traditional address information? Can data be updated/fixed during import, or will you have to abort, fix and retry? Is there a template that needs to be followed? What happens if an error occurs with just some of the records? Were the rest imported? Users, and even developers, rarely think about these things until it is time to import the data. Guidance on the above topics can certainly frame the import process for the client. However, guidance only goes so far - it puts the onus on the customer to interpret the instructions and to ensure the data matches the instructions. Should the user have to watch a 15-minute tutorial video or read several pages of instructions to perform an import? Should the user have to spend an hour combing through a CSV or spreadsheet file to ensure it matches the instructions provided? If an error occurs, should the user be expected to troubleshoot?  As you can see, even the clearest guidance can still result in support tickets or calls. Unclear guidance can cause no end of grief. Unintelligible Errors When the import fails, the user needs to know why. A trial and error approach can result in a great deal of frustration, and unnecessary work, particularly when the issue should be simple to fix. When you are dealing with a lot of records, it can be hard to communicate issues back to the user. If an import fails, and you receive the following error, would you know what to do next? The below error was received by a user of the thirtybees e-commerce platform, and posted to . As you can see, there is not much to assist the user in this message. their forums Here is another similar situation from . In this case, note that the directions suggest reviewing the logs to determine what the problem is, and investigate the error. It also suggests that the data may be in an unstable state. Confluence/Jira Did the data get corrupted in the import process? The error does not state that the data is definitely corrupt, but the implication that the importer could have ruined your data does not exactly inspire confidence. Do you want to go through the effort of configuring the data in a CSV just to risk losing it in another attempt? As you can see, there is a prevalent expectation with many in-house importers that the user should investigate, then troubleshoot the issue. There is also a sense that the issue is the customer's error, not a problem with the user experience of the website. If this was your first experience importing a spreadsheet or CSV for a product, would you continue, or would you look for a solution that worked better? One in which the directions and the error messages both were clear and upfront? Data Transformation Issues Transforming data from one system to another is a hard exercise. It is highly likely that whatever system you are exporting data from has different field names or data structures. If a user can’t easily transform data into a format that fits your systems schema, they are bound to be frustrated. For example, let’s say you export data from a system that has an “Address” field, but in your own system you split address fields into Street Address, City, State, Zip Code, Latitude, and Longitude. This is a complex situation to deal with. You will likely have to use an external API to geocode addresses from the old system so that you can transform and map the data to the columns in your own database. Data Mapping Issues There are numerous challenges to mapping fields in a CSV or spreadsheet for import. Quite commonly, the limitations of the importer can be the biggest challenge. For example, let’s look at an example of a product with their own importer. is an email campaign product competing with products such as Mailchimp. They offer a fairly strong import solution considering it is not their core strength. However, you’ll notice quite quickly that the fields they recognize are quite limited. For example, after creating a simple test CSV and importing it, only one field is recognized, the rest are set to ignore. EmailOctopus You’ll note that the headers in the CSV are fairly standard. The Name field provides a first and last name, and from the prompt on the second column, we can surmise that they wish these to be split into separate fields. Zip Code is not recognized, nor is Company. It does allow you to map these fields to acceptable fields within the system. Company, for example, should prompt with Company Name as an option, and it does not. In fact, it offers the same options for all three, without any consideration of the content of the fields or the header name. The user should not have to work too hard to fit their data into your organization’s parameters. New columns should not have to be created for commonly used columns, particularly those that do likely have an on-site equivalent, such as Company Name for Company. This can produce additional work within the product itself if what amounts to duplicate columns need to be cleaned up. Mapping columns from an old system to your own should be an easy, simple experience. Product, Development, and Sales Resource Constraints Creating a great CSV import experience typically isn’t the focus of your product. Furthermore, sales teams often need expensive developers to sidetrack themselves from important features in order to onboard customers. Getting sales-related data into a system is critical to company growth and minimizing the sales cycle. How Flatirons Fuse Provides a Seamless CSV Import Experience Now that we’ve outlined some of the key problems that can result in a poor import experience, let’s look at a solution, shall we? Flatirons Fuse offers a solution designed specifically to help businesses stick to their core strengths, to focus on improving their product, their service, and their expertise. Let’s look at how we help and frame our capabilities in relation to the problems mentioned above: Clear Guidance A clean, simple, and direct interface is key to a positive user experience. Potential customers should not need to navigate between documentation and your onboarding site to get through the import process. Our interface provides the critical information necessary to get their data imported into your system by default. As you can see, the customer is immediately provided the file types supported by our importer. Required headers are clearly identified, as well as optional headers. This guidance ensures the customer knows exactly what is needed. The information here is customized based on templates that the client can manage and create for whatever the system might be. For example, the required headers for a web-based financial application might be very different from a CRM. But from the importer's perspective, it is clear, simple and easy. Once you know the file requirements, a simple drag and drop interface and an option to browse for your files are the de facto standard. The directions are simple, the process just as simple. Mapping Imported Data Once your customer’s file is uploaded, the process remains just as clear and simple. The next step is simply to identify and confirm that there’s a header row so that we can perform column mapping. Finally, we can verify the results. You may have noticed above that while the system accepts Address as a field, the user uploaded a file that provides Location instead. This is where manual column mapping comes into play. Note that while it says that we matched columns correctly, we have a duplicate in the results. Our AI-powered column matching correctly mapped all columns, with the exception that Email Address was mapped to Address. So, we have a duplicate column, and a missing column…start over, right? Not at all. Simply click the X on the second Email Address column in Your Columns. In the drop-down, select Location from the headers available in your CSV file. An easy fix, and we are good to go. Click Continue, and you’ll see the results of this minor change. As you can see, the information is now associated with the approved headers/columns for the target system, even with a mid-stream correction. Now we see that there is another small issue. Our validations have automatically flagged one of the emails provided as not being formatted correctly (missing the @ symbol). Simply click in the box, and correct the error. With clear direction throughout, AI helping to match headers (even if not always correct), and the capacity to make necessary changes even while uploading, the customer will have no problem importing the data to your system. All that remains is to submit the data. Fixing Backend Import Errors If there is an issue in importing the data, our validation system will kick into gear, based on criteria provided by you, our client. Error messages can be customized to provide the info the customer (the importer) requires to resolve whatever the issue may be. The below error message is simply a test message to illustrate the function: Upon receipt of the error, simply click OK, and the problem will be highlighted in the results area. As you can see, the system detected the error and highlighted it clearly for the user. In this case, we are told the entry is a duplicate value. We know exactly what the problem is, and have a good idea of how to fix it. So we change the entry, or delete it (if it was a true duplicate), and we’re ready to try again. Looks like a success to me. Your customers will have clear instructions to set up the import process, clear guidance throughout the process, plus true validations to help deal with any errors that come up. All without the need to spend countless hours developing an importer that may not meet these requirements, resulting in frustrations for both customers and support. The Bright Side - A Free and Powerful CSV Importer Exists Why call out the bright side when the preceding content is pretty bright to begin with? Well, because there is no dark side, of course. In case it isn’t clear, the side we will now cover is the client-side - how easy Flatirons Fuse is to configure, our white-label and customization options, and that it is free. Not only free to try, but also free forever, providing your organization doesn’t exceed 10000 records per month. So when we say you get more than you pay for, we clearly mean it. Getting Started for Free Fuse has a tier that is free for life. So you want to get started, or at least see what the fuss is about. Simply click Sign in from . our website This will take you to our Login/Registration page. Sign in with your email address or Github account if you are registered, or click Sign Up if not. Fill in the form, or register via github (in which case your account details will be filled in from Github). Once registered and logged in, we make it easy to get started immediately, by creating your first template. Our Get Started wizard gets you underway immediately. Simply click Create Template and you are underway. Creating the Template Upon clicking Create Template, you are prompted to provide a name for the template. This should reflect its purpose. You can create multiple templates for multiple applications/sites if necessary. But we’ll start with just one. Next, we’ll need to add some columns. Here you can see how easy it is to add columns. Make your selections and remember to select whether the columns are required or not. Note that there is a great deal of flexibility in the type of data that can be included, such as boolean (yes or no) values, string (characters), integers (whole numbers) structured data, and more. Simply select the best fit for your needs for each column. This flexibility is one of the key reasons that Flatirons Fuse can meet the CSV import needs of so many organizations. Initializations, Data Transformations, and Hooks Validations and data transformations are not limited to those we provide by default, like email format. Custom validations for your organization can be enforced using . These additional validations can be applied very simply with a few lines of code. For example, you could implement a function that auto-capitalizes the first letter of a column named City, removing the onus from the user. Or, you can implement a validation that Zip Code must be limited to a grouping of five numbers. The guidance linked above can help you set any number of custom validations, or what you need and we’ll help you set it up. initializations and hooks let us know Testing the Template That’s all there is to creating a template. It is now ready for testing. We provide the means to test your template within Flatirons Fuse, and guides you on the steps to do so. Follow the guidance to test the solution on the platform that best fits your use case. our documentation A White-Labelled CSV Importer A Software as a Service solution, particularly one designed to serve as part of your onboarding process, isn’t much of a solution if it does not look like part of your stack. The idea of our importer is to blend and become a seamless part of your user experience. To that end, our solution is completely white-label and offers numerous customization options to integrate with your branding. In fact, you can see how to integrate these options in the third step of the Get Started Guidance. Upon selecting customize importer, you will be taken to our customization page, where you can provide your company logo, and specify the color scheme you wish to use to make our importer your own. Within just a few short minutes, you can have a basic template setup, test it, and customize it to fit your product or site. Free is Good Oh, and did we mention it’s free? Because it is. Our pricing system is clearly outlined on if you want the full details, but the gist of it is that our solution will not cost you a thing until you begin to exceed 10,000 records per month. , our website Summary By leveraging our own singular focus on improving your onboarding and CSV/spreadsheet data import process, you can deliver on your full organizational potential without worrying about sideline development efforts, documentation of a problematic process or support calls at the outset of your onboarding process. You can focus on whatever you do best, while we handle what we do best.

Implementing a Powerful CSV & Spreadsheet Import Experience

You might have heard about . We use them in everyday development. Spread Operators Definiti on: According to MDN Web Docs Spread syntax (...) allows an iterable such as an array expression or string to be expanded in places where zero or more arguments (for function calls) or elements (for array literals) are expected, or an object expression to be expanded in places where zero or more key-value pairs (for object literals) are expected. Use case scenario: We will see this by comparing the normal arrays method, so this can be helpful to everyone including those who didn't use it as well as those who are also familiar with it. String to Array with Spread: Just spread the variable with and get the array. … const myName = "Jhon Doe"; 
const convertMyNameToArray = [...myName]; 
console.log(convertMyNameToArray); 
//Output: Array (8)[ J,h,o,n, ,D,o,e ] Spread for Merging Array: Here spread both the array as and get one single array [..arr1, …arr2] const array1 = [50,150,250]; 
const array2 = [100,200,300]; 
const mergedArray = [ ...array1, ...array2 ] 
console.log(mergedArray); 
//Output: Array(6)[ 50,150,250,100,200,300 ] Cloning Array using Spread: Cloning of an array can be done in various ways. Here is a demo. The demo shows both ways; with and without spread operators: //Without Spread Operator: 
const animals = ['lion','tiger','zebra']; 
const wildAnimals = animals;
wildAnimals.push('elephant')
console.log(animals);
//Output: Array (4)[ lion,tiger,zebra,elephant ]

//Here original array is affected although we pushed in cloned array.

//With Spread Operator:
const animals = ['lion','tiger','zebra'];
const wildAnimals = [...animals];
wildAnimals.push('elephant');
console.log(animals)
//Output: Array (3)[ lion,tiger,zebra ]
//Here original array is NOT affected Do you know why it behaves like this? Stay tuned I am bringing another blog for explaining this. Why separate blogs? because it is required to understand the concepts of data types and it’s out of the context of this blog. In short: The spread operator . When you have nested data in an array or object the spread operator will create a deep copy of the topmost data and a shallow copy of the nested data. makes deep copies of data if the data is not nested A of an object is a copy whose properties share the same references (point to the same underlying values) as those of the source object from which the copy was made. As a result, when you change either the source of the copy, you may also cause the other object to change too Shallow Copy: shallow copy A of an object is a copy whose properties do not share the same references (point to the same underlying values) as those of the source object from which the copy was made. As a result, when you change either the source of the copy, you can be assured you're not causing the other object to change too; that is, you won't unintentionally be causing changes to the source or copy that you don't expect. Deep Copy: deep copy Set Object to Array: //Creating a new Set Object 
 const flowersSet = new Set(['rose','lotus','lilly']); 
 console.log(flowersSet); 
 //Output: Set (3) { rose=> rose,lotus=> lotus,lilly=> lilly }
 
//Converting the Set Object to Array
const newFlowerArray = [...flowersSet];
console.log(newFlowerArray);
//Output: Array (3)[ rose,lotus,lilly ] Nodelist to Array: //create nodelist object 
const h1s = document.querySelectorAll('h1');
//convert nodelist to an array
const h1sArray = [...h1s] Min or Max value from an array: //USING APPLY 
const ages = [21,52,55] 
const elderPerson = Math.min.apply(Math,ages); //21 
const younderPerson = Math.max.apply(Math,ages); //55

//USING Spread
const elderPerson = Math.min(...ages); //21
const younderPerson = Math.max(...ages); //55 Spread Operator for Objects: const user1 = { name: 'Jhon', age: 21, };
const user2 = {
  name: "Doe",
  dob: "5th Jan 1990"
};

const mergedUsers = {...user1, ...user2};
console.log(mergedUsers)
//Output: {name: 'Doe', age: 21, dob: '5th Jan 1990'} Cheers! This article was first published . here

Javascript Cheatsheet: Spread Operators 

Highlights Check Point Research (CPR), in collaboration with Kaspersky’s Global Research & Analysis Team (GReAT), have been tracking an ongoing attack targeting a small minority group of Uyghur individuals in Xinjiang and Pakistan Attackers use fake United Nations (UN) documents and human rights websites to spread malware that has the ability to exfiltrate information and take control of victims’ PCs The Uyghurs are a Turkic ethnic group, culturally affiliated with Central and East Asia, and considered one of China’s 55 officially recognized ethnic minorities Background In the past year, Check Point Research (CPR), in collaboration with Kaspersky’s Global Research & Analysis Team (GReAT), have been tracking an ongoing attack targeting a small group of Uyghur individuals located in Xinjiang, China and Pakistan. Malicious actors disguised their attacks in the following ways: They created documents that appear to be from the UN, using real UN information to ensure these looked authentic. Set up websites for non-existent organizations claiming to fund charity groups This blog details the investigation of the decoy methods this group used. Fake UN documents as a tool for initial infections The researchers’ investigations began with a malicious document found on the free malware scanning service VirusTotal named “ ” which carried the logo of the United Nations Human Rights Council (UNHRC), and contained content from a UN general assembly discussing human rights violations that made the document seem genuine. UgyhurApplicationList.docx After the user opens the document by clicking on “enable editing”, a malicious external template containing a macro code is downloaded, and this macro code proceeds to decode an embedded backdoor. After the backdoor is decoded, it is then named “OfficeUpdate.exe” and saved under the %TEMP% directory. In the two “OfficeUpdate.exe” examples the researchers located, the payload was a shellcode loader that utilizes basic evasion and anti-debugging techniques by using functions such as sleep and QueryPerformanceCounter. Delivery Websites – Impersonating the UN’s Commission for Human Rights The domain observed in the malicious document (officemodel[.]org) led to the same IP address as unohcr[.]org – a domain impersonating the Office of the High Commissioner for Human Rights (OHCHR). Investigating this method of fake domains and websites revealed a tactic of distributing malware through fake websites that host malicious executables targeting Windows users. Another IP address that unohcr[.]org led to was a domain named tcahf[.]org, which hosted a website claiming to represent the Turkic Culture and Heritage Foundation (TCAHF). TCAHF claims to be a private organization that funds and supports groups working for Turkic culture and human rights, when in fact it is a made up entity, with most of its website’s content having been copied from the legitimate “ ”. opensocietyfoundations.org The malicious functionality of the TCAHF website is well disguised and will only appear when the victim attempts to apply for a grant (see the added “Application” menu button in Fig. 5). The website then claims it must make sure that the operating system of the victim’s PC is safe before they enter sensitive information for the transaction, and asks them to download a program to scan their PC environment. The website offers two download options, one for MacOS and one for Windows, but when the team analyzed the website, only the download for Windows was available, while the MacOS version link served an empty file. Uyghur minority as a target Based on the nature of the malicious websites and the decoy content used in the delivery document, the researchers assessed that this campaign is intended to target the Uyghur minority or the organizations supporting them. The Uyghurs are a Turkic ethnic group, culturally affiliated with Central and East Asia, and considered one of China’s 55 officially recognized ethnic minorities. The research team’s telemetry supports this assessment, as it has identified a handful of victims in Pakistan and China. In both cases, the victims were located in regions mostly populated by the Uyghur minority. Attribution Although the researchers were unable to find code or infrastructure similarities to a known threat group, they attribute this activity, with low to medium confidence, to a Chinese-speaking threat actor. When examining the malicious macros in the delivery document, the research team noticed that some excerpts of the code were identical to VBA code that have appeared in multiple Chinese , and might have been copied from there directly. forums Impact of attacks and conclusion While most of the activity described above happened in 2020, it appears the attackers behind this campaign are still active, and working with newly registered domains. The findings of this research indicate these attacks are ongoing, and new infrastructure is being created for what looks like future attacks. Most recently, one of the domains appeared to be impersonating the Turkic Ministry of the Interior, but this and another domain redirected to the website of a Malaysian government body called the “Terengganu Islamic Foundation”. This suggests that they are pursuing additional targets in countries such as Malaysia and Turkey. However, the malicious group might still be developing those resources, since researchers have yet to find any malicious artifacts associated with those domains. The malicious executables created by the attackers exfiltrate basic information about the infected system but can also download a second-stage payload, or in the case of the documents, fetch additional commands from the command and control server. This means that the researchers have not yet seen all the capabilities of this malware, or the full course of action taken by the attackers following a successful infection. The motivation behind these cyberattacks seems to indicate a campaign of espionage, with the end game of the operation being the installation of a backdoor into the computers of high-profile targets in the Uyghur community. The attacks are designed to fingerprint infected devices, including all of its running programs. CPR and Kaspersky GReAT researchers will continue investigating this issue and report any new relevant findings.

How United Nations Documents and Human Rights Websites Were Used in Spreading Malware

Research by: Aviran Hazum, Bodgan Melnykov & Israel Wenik Overview Check Point Research (CPR) recently discovered malware on Google Play hidden in a fake application that is capable of spreading itself via users’ WhatsApp messages. If the user downloaded the fake application and unwittingly granted the malware the appropriate permissions, the malware is capable of automatically replying to victim’s incoming WhatsApp messages with a payload received from a command-and-control (C&C) server. This unique method could have enabled threat actors to distribute phishing attacks, spread false information or steal credentials and data from users’ WhatsApp accounts, and more. General As the mobile threat landscape evolves, threat actors are always seeking to develop new techniques to evolve and successfully distribute malware. In this specific campaign, Check Point’s researchers discovered a new and innovative malicious threat on the Google Play app store which spreads itself via mobile users’ WhatsApp conversations, and can also send further malicious content via automated replies to incoming WhatsApp messages. Researchers found the malware hidden within an app on Google Play called ’FlixOnline.’ The app is a fake service that claims to allow users to view Netflix content from all around the world on their mobiles. However, instead of allowing the mobile user to view Netflix content, the application is actually designed to monitor the user’s WhatsApp notifications, and to send automatic replies to the user’s incoming messages using content that it receives from a remote command and control (C&C) server. The malware sends the following response to its victims, luring them with the offer of a free Netflix service: “2 Months of Netflix Premium Free at no cost For REASON OF QUARANTINE (CORONA VIRUS)* Get 2 Months of Netflix Premium Free anywhere in the world for 60 days. Get it now HERE https://bit[.]ly/3bDmzUw.” Utilizing this technique, a threat actor could perform a wide range of malicious activities: Spread further malware via malicious links Stealing data from users’ WhatsApp accounts Spreading fake or malicious messages to users’ WhatsApp contacts and groups (for example, work-related groups) Extort users by threatening to send sensitive WhatsApp data or conversations to all of their contacts Technical Analysis When the application is downloaded from the Play Store and installed, the malware starts a service that requests ‘Overlay’, ‘Battery Optimization Ignore’, and ‘Notification’ permissions. The purpose behind obtaining these permissions is: Overlay allows a malicious application to create new windows on top of other applications. This is usually requested by malware to create a fake “Login” screen for other apps, with the aim of stealing the victim’s credentials. Ignore Battery Optimizations stops the malware from being shut down by the device’s battery optimization routine, even after it is idle for an extended period. The most prominent permission is the Notification access, more specifically, the Notification Listener service. Once enabled, this permission provides the malware with access to all notifications related to messages sent to the device, and the ability to automatically perform designated actions such as “dismiss” and “reply” to messages received on the device. After the permissions are granted, the malware displays a landing page it receives from the C&C server and immediately hides its icon so the malware can’t be easily removed. This is done by a service that periodically contacts the C&C and updates the malware’s configuration accordingly. The service can achieve these goals by using multiple methods. For instance, the service can be triggered by the installation of the application and by an Alarm registered in the BOOT_COMPLETED action, which is called after the device has completed the boot process. The response from the C&C contains a configuration with the following field: Field Purpose landing_page A URL to display to the victim after permission granting. message_inbox The message to send as a reply to all incoming messages. message_limit Unused, potentially could indicate as an “upper limit” for the amount of messages to send out. delay Unused. url Unused. delay_browser Delay before showing popup with specific URL. enable_browser C&C check flag. enable_webview Indicates which app to use to open the URL. webview_url The URL for the WebView popup activity. browser_url URL for the browser popup. Once this is complete, the malware has everything needed to distribute the payload. With the OnNotificationPosted callback, the malware checks for the package name of the originated application, and if that application is WhatsApp, it will process the notification. Responsible disclosure Check Point Research responsibly notified Google about the malicious application and the details of its research, and Google quickly removed the application from the Play Store. Over the course of 2 months, the “FlixOnline” app was downloaded approximately 500 times. Conclusion This wormable Android malware features innovative and dangerous new techniques for spreading itself, and for manipulating or stealing data from trusted applications such as WhatsApp. It highlights that users should be wary of download links or attachments that they receive via WhatsApp or other messaging apps, even when they appear to come from trusted contacts or messaging groups. If a user was infected, they should remove the application from their device, and change their passwords. Stay protected from mobile threats Check Point Harmony is the Mobile Threat Defense (MTD) solution, providing the widest range of capabilities to help you secure your mobile workforce. Mobile market-leading Harmony Mobile provides protection for all mobile vectors of attack, including the download of malicious applications and applications with malware embedded in them. Appendix 1 – IOCs FlixOnline – 1d097436927f85b1ab9bf69913071abd0845bfcf1afa186112e91e1ca22e32df C&C – netflixwatch[.]site Package Name – com.fab.wflixonline Certificate – BEC2C0448558729C1EDF4E45AB76B6A3EE6E42B7

How a Wormable Android Malware Used to Spread Using WhatsApp Auto-Replies

This article will cover the and how you can use them in your day-to-day JavaScript programming. This article assumes you have some familiarity with coding in the JavaScript ecosystem. spread operator Prerequisites We will be using a code example from a previous post - . Since the spread operator does the opposite thing of what do, a good understanding of would be helpful here. However, as a brief introduction, using the three periods syntax , rest parameters allow us to capture an indefinite number of function arguments as an array. It is kind of similar to or . How to Use Rest Parameters in JavaScript rest parameters rest parameters (…) Java Varargs C# parameter arrays In this code example, the log function takes the and parameters then outputs them to the console. level args { ( i = ; i < args.length; i++) { .log(level, , args[i]);
  }
} // ------------------ 👇 a rest parameter ( ) function log level, ...args for var 0 console `{$i}` The Spread Operator or Spread Syntax Basically, the spread operator converts an array to a list of comma-separated values (CSV), contrasting rest parameters, where it takes a CSV and converts it into an array. We have below a logging function similar to the one above (but without using rest parameters). Let us assume that the variable value comes from an external API that returns a list with varying types and values, and we want to log each element to the console. We can see from the output that while the function is expecting four arguments, we’ve only given it two. Hence the being shown twice for variables and . responseBody undefined arg2 arg3 { .log(level, , arg1); .log(level, , arg2); .log(level, , arg3);
} responseBody = [
  { : }, , ];

log( , responseBody); ( ) function log level, arg1, arg2, arg3 console '{0}' console '{1}' console '{2}' var name 'Non-Master Anakin Skywalker' 1 false 'DEBUG' // expected output: // DEBUG {0} [ { name: 'Non-Master Anakin Skywalker' }, 1, false ] // DEBUG {1} undefined // DEBUG {2} undefined This can be resolved by providing the correct arguments to the function. This can be done by getting each element by index and pass each to the function. log( , responseBody[ ], responseBody[ ], responseBody[ ]); 'DEBUG' 0 1 2 // expected output: // DEBUG {0} { name: 'Non-Master Anakin Skywalker' } // DEBUG {1} 1 // DEBUG {2} false That should be good, our logging function is working as expected, but this can be improved. The desired output can be done with less code using the . We don’t need to access each list element by index. We can use the three periods syntax again here and prepend it to the list variable. Converting it to a comma-separated values that the logging function is expecting. spread operator (…) log( , ...responseBody); // --------- 👇 spread operator 'DEBUG' // expected output: // DEBUG {0} { name: 'Non-Master Anakin Skywalker' } // DEBUG {1} 1 // DEBUG {2} false What was done by using the spread syntax was taking a list and converting it to a CSV. When we used the logging function at first, we missed providing the remaining two arguments. As if we were calling the function like this: log( , [{ : }, , ], , ); 'DEBUG' name 'Non-Master Anakin Skywalker' 1 false undefined undefined Using the spread operator, we have correctly provided the logging function’s arguments. the value of the list into an actual CSV that is accepted by the function. As if we were calling the function like this, which is the correct usage: Spreading log( , { : }, , ); 'DEBUG' name 'Non-Master Anakin Skywalker' 1 false Spread Operator and Rest Parameters If you’re already comfortable with the concept of the spread operator, let us go back to my previous point where it is conceptually the opposite of rest parameters. Let’s use the first logging function from above that uses and use it together with the spread operator. rest parameters When directly passing the list variable, it shows only one debug output to the console. This is expected behavior since what rest parameters do is capture an indefinite number of arguments supplied to a function. In this case, we’ve only given it one argument, thus logging only once. { ( i = ; i < args.length; i++) { .log(level, , args[i]);
  }
} responseBody = [
  { : }, , ];

log( , responseBody); ( ) function log level, ...args for var 0 console `{ }` ${i} var name 'Non-Master Anakin Skywalker' 1 false 'DEBUG' // expected output: // DEBUG {0} [{ name: 'Non-Master Anakin Skywalker' }, 1, false] When passing the list variable using the spread syntax to the function, it shows three debug console outputs, which is the expected output, matching the number of elements from the list. log( , ...responseBody); 'DEBUG' // expected output: // DEBUG {0} { name: 'Non-Master Anakin Skywalker' } // DEBUG {1} 1 // DEBUG {2} false This shows how rest parameters and the spread operator are opposite of each other and how the two can be used together. Let us look at another example of the spread operator but with a list with four elements. This time it will output to the console four times. responseBody = [
  { : }, , ,
  [ , ]
];

log( , ...responseBody); var name 'Non-Master Anakin Skywalker' 1 false 'Tatooine' 'Coruscant' 'DEBUG' // expected output: // DEBUG {0} { name: 'Non-Master Anakin Skywalker' } // DEBUG {1} 1 // DEBUG {2} false // DEBUG {3} [ 'Tatooine', 'Coruscant' ] Adding Element to a List There is also another use for the spread operator aside from passing them as function arguments. Here we have a list variable that has two elements. If we want to add a new element at the end of the list, that should be straightforward. tail head = { : }; tail = [{ : }, { : }]; .log(tail); tail.push(head); .log(tail); const value 'A' const value 'B' value 'C' console // expected output: // [ { value: 'B' }, { value: 'C' } ] console // expected output: // [ { value: 'B' }, { value: 'C' }, { value: 'A' } ] When adding a new element at the front of the list, we can use the spread operator. head = { : }; tail = [{ : }, { : }]; newList = [head, ...tail] .log(newList); const value 'A' const value 'B' value 'C' // -------------------- 👇 spread operator const console // expected output: // [ { value: 'A' }, { value: 'B' }, { value: 'C' } ] End In this post, we looked at the spread operator in order to convert a list into a CSV. We learned how the spread operator is the opposite of the rest parameters and how they can be used together. Then, we concluded by using the spread operator to append elements at the front of a list. Also Published On: https://micogongob.com/how-to-use-the-spread-operator-in-javascript

Stories, News & Mentions about SPREAD

Spread’s Stories on HackerNoon