Work to rescue an orca calf stuck in a lagoon on Vancouver Island is ramping up, with First Nations and experts finalizing a plan.

B.C. orca rescue plan being finalized

The Department Of Fisheries and Oceans (DFO) is still planning to lift a stranded orca calf out of Little Espinosa Inlet near Zeballos.

Plan to lift B.C. orca calf out of remote lagoon still in the works

ZEBALLOS, B.C. — A Vancouver Island First Nation that has adopted a young killer whale as its newest family member says concern rises about its ordeal with each day that passes, but strong hope for a rescue remains. The Ehattesaht First Nation says in a statement people have been listening ...

Stranded B.C. calf's calls 'make you almost weep,' says Ehattesaht First Nation

ZEBALLOS, B.C. — Plans are settling into place for an attempt to rescue a young killer whale calf stranded in a tidal lagoon near the Vancouver Island village of Zeballos within four or five days, says the area&#x27;s First Nation chief.

Chief says rescue effort for stranded orca calf four, five days away as plans ramp up - Victoria Times Colonist

The skin of a young killer whale trapped in a Vancouver Island lagoon is turning white due to the low salinity of the water, but the Fisheries Department says the calf is active and isn&#x27;t showing signs of emaciation.

Trapped B.C. orca calf's skin whitening, no sign of emaciation: Fisheries Department

It’s been 17 days since a Big’s killer whale was beached and drowned in a lagoon near Zeballos while teaching her calf to hunt.

Photo released of sandbar blocking orphaned orca calf from escaping lagoon near Zeballos

ZEBALLOS, B.C. — The skin of a young killer whale trapped in a Vancouver Island lagoon is turning white due to the low salinity of the water, but the Fisheries Department says the calf is active and isn&#x27;t showing signs of emaciation. The statement issued late Monday says the female calf&#x27;s ...

ZEBALLOS, B.C. - The skin of a young killer whale trapped in a Vancouver Island lagoon is turning white due to the low salinity of the water, but the Fisheries

Trapped B.C. orca calf’s skin whitening, no sign of emaciation: Fisheries Department

10:07 AM: Eclipse visibility: Iffy. Whale visibility: Good news - Kersti Muul just sent word of two groups of orcas in the area:

West Seattle Blog… | PHOTOS: Two groups of orcas in the area

National Geographic Senior Editor Christine Dell’Amore discusses the rogue orcas thriving in the high seas, living far offshore California and Oregon. Visit natgeo.com for more coverage. This segment aired on KTLA 5 Weekend Morning News on April 7, 2024.

NatGeo on why rogue orcas are thriving in high seas | KTLA

OCRA

ocra

Data entry can often turn tedious and inaccurate if done manually. You can avoid this cumbersome process with an automated optical character recognition software. What OCR does is, it uses machine learning to scan each character on a page individually. It allows the documents to be uploaded as text documents instead of images. Any kind of printed document, be it receipts, invoices, contracts, utility bills and much more can be scanned easily and promptly with this solution. Data extraction steps using Infrrd OCR How is Infrrd’s OCR different from Traditional OCR? Our OCR solution is different in two ways: We don’t use templates but NLP for recognizing entities in a document. This helps us identify business names, bank details, amounts, prices etc. irrespective of where they are placed in the document. Read more on why templates are a bad idea. here We use Machine Learning to fix what traditional OCR’s cannot figure out, by building an automatic context around the document being handled. Watch to understand how Infrrd OCR solution scans in a fine-tuned manner. this video How does Infrrd OCR work? A unique API key is generated for you when you first integrate our optical character recognition software with your mobile or desktop. Even for a trial use, our representatives offer you this key so that you’re all set to scan your documents. Step 1: Upload any document in the format of PDF, JPEG, TIFF, PNG etc. to the software. You can use Infrrd’s software as a white-labeled mobile app, on your desktop or as a cloud solution to start scanning. Step 2: Our software starts extracting line items and other key fields such as logo, expense type, merchant name, date of the transaction, amount, currency VAT/GST, business name etc. We can even customize it to extract any other information that you might need. The OCR software provides character-level and word-level confidence scores. These scores are indicators of whether the OCR software believes the extracted information to be accurate. Step 3: The extracted data is made available to you in formats like XML, CSV, JSON etc. as per your requirement. Step 4: What add-ons do you get with OCR Text Recognition Solution by Infrrd ? Unlike other OCR solutions, Infrrd OCR doesn’t have a fixed template. You don’t need to manually fill in the details left out by your regular image text recognition tool. Our solution uses AI to understand the specific details in your invoices, receipts, bank statements or even handwritten forms and extracts them in a defined way. Self-learning software: It can comprehend documents in French, Dutch, German and Spanish apart from English. Recognizes major world languages: Whether you’re sitting in your office or you are in transit, this invoice scanning software is available via cloud solution and mobile app. User-friendly in a remote location as well as on the go: Infrrd OCR doesn’t follow a set template. It identifies line items and key fields like tax, amount, quantity, business name, and other details automatically which might have been left out by the other OCR tools. Freedom from templates: Be it finance, education, insurance, retail, logistics or warehouse, this document extraction application for enterprises can be applied to any sector. Wherever you need a little help in processing humongous chunks of data, our solution will be there to aid you. Functional across different business verticals: So, why wait? Get a of Infrrd OCR for your invoices, text or images now and organize your data! Our support staff will guide you through every step. free demo Ready, set, scan! Originally published at infrrd.ai on April 17, 2018.

What is OCR: A Step By Step Guide For Data Extraction

Topographical Notes

FOOTPRINTS OF COLONIZATION

Table of Contents Introduction The objective and need for the CLIP algorithm The working procedure of the CLIP algorithm Limitations Realtime Applications Introduction CLIP aka “Contrastive Language Image Pre-training” is one of the renowned algorithms discussed in a white paper named “Learning Transferable Visual Models From Natural Language Supervision” published by researchers at OpenAI - An artificial intelligence research laboratory. The major consumption of CLIP is done in the use cases based on computer vision that uses an algorithm named “Dall-E 2” which was also developed by the Open AI team. More precisely, CLIP is being used as a helper model for the “Dall-E 2” algorithm. But don’t misunderstand that CLIP is not powerful as it is consumed as a helper model :) Despite being a helper model, CLIP is considered an important step in deep learning research. We can implement CLIP for separate problem-solving even without “Dall-E 2”. In this article, we can discuss the objective, working procedure and some of the pros & cons of the CLIP through some real-life examples and how we can simplify the life of our deep learning projects. The objective and need for the CLIP algorithm The primary objective behind the CLIP algorithm is to find a particular text from a list of texts which is more similar to the given image. For example, Let us consider the following image as input- And let's say we have some of the texts in a given list- A photo of an airplane. A photo of a bird. A photo of a bear. A photo of a giraffe. A photo of a car. The primary task of a CLIP model is to match the most appropriate text from the given list to the input image as shown below- Fundamentally, this is an artificial neural network that considers each text in the list as a class and assigns a probability value for each text corresponding to the image. Logically, the text that gets the maximum probability value can be considered as the output. One big positive aspect of the CLIP is that it already knows all the words in the “English” language. Some of the characteristics of the CLIP model that makes it special when compared with other similar algorithms are- The CLIP model is not restricted to a single word in the text. Instead, it tries to extract every piece of information from all the words in the input sentence and all the pixels of an image. It never forgets to remember all the aspects of an input image such as the objects in the background, colour, shapes etc. For example, Let us consider the following input image- All of the texts in the given list except the last one look like a logical match for the input. Any other model would have struggled to reach a high conviction probability value for a particular class. However, the CLIP would have analysed the patterns for all the aspects in this image such as the kennel, cell, dog etc. The sunlight seems to be coming from outside to inside. Hence, it should be an indoor structure. Also, there is a presence of an animal instead of a human. Hence, it should not be a Jail but could be a Kennel. Such kind of advanced analysis considering all the aspects of the image & text might not be possible for other models in the same league. CLIP has also been seen working well in Optical Character Recognition use cases which is not possible in similar models- Other branches of computer vision such as geo localisation, facial emotion recognition etc also seem to be working well with the CLIP model. CLIP has shown great zero-shot performance even in unseen inputs. Also, it doesn't expect the text to be in a single word such as “cat”,” dog”,” bird”,” car”.” plane” etc. Instead, it can comprehend a group of words logically framing a sentence. This feature is not generally seen in other models. The working procedure of the CLIP algorithm The CLIP algorithm has been trained on 400 million images with paired text descriptions that make it highly knowledgeable about the universe and confident in solving complex tasks with complex images and texts. The Imagenet dataset consists of only 1.2 million images. 400 million is almost 300 times that of 1.2 million.  Most of the 400 million images are directly scraped from the internet which makes it a highly diversified and large collection that increases its pattern detection capability. For the development of the CLIP architecture, we need to encode both the images and corresponding texts into mathematical vectors. This is because a machine learning algorithm won't able to infer the information if it's in a visual or textual format. Hence, we need to convert them into numerical values. Image input is converted into a mathematical vector using a Transformer or Resnet algorithm The textual input is into a mathematical vector using a Transformer algorithm- Since we have a list of Image-Text pairs, we need to denote it using certain alphabets. Each image is denoted as I1, I2,I3...IN etc. Each text is denioted as T1,T2,T3…TN etc. After that, we need to build a similarity matrix with each of the images as rows and each of the texts as columns. As mentioned in the above image, the diagonal image-text pairs will have more similarity as they are referring to the same context. The non-diagonal elements are random pairs that do not belong to the same context. Hence, their similarity value will be low. The goal of the optimisation functions will be to increase the similarity value for the diagonal pairs as much as possible and decrease the similarity between non-diagonal image-text pairs. At one point in learning, the model will be able to learn the hidden patterns that match the images and texts that belong to the same context and differentiate the images and texts that belong to different contexts. This procedure is technically called . “Contrastive pre-training” CLIP is considered a “computationally efficient” algorithm. This is because they use the transformer algorithm to encode images and texts which access the data in a parallel fashion. If we use other algorithms such as LSTM or RNN, they tend to access the data for encoding in a serial fashion which might consume a lot of time and space. Since the CLIP can match an image with a long sentence, the researchers usually create a text prompt something like “A photo of a _____ “. Then, while iterating through a list of texts, the computer program automatically fits every single word from the list into this text prompt like- A photo of a plane A photo of a car A photo of a dog etc. This text is then encoded and matched with the encoded vector of the input image for calculating the similarity value. Limitations Challenges related to scalability On datasets with training splits, the performance of zero-shot CLIP is on average competitive with the simple supervised baseline of a linear classifier on top of ResNet-50 features. On most of these datasets, the performance of this baseline is now well below the overall state of the art. Significant work is still needed to improve the task-learning and transfer capabilities of CLIP. While scaling has so far steadily improved performance and suggests a route for continued improvement, researchers estimate around a 1000x increase in computing is required for zero-shot CLIP to reach overall state-of-the-art performance. This is infeasible to train with current hardware. Further research into improving the computational and data efficiency of CLIP will be necessary. Challenges related to zero-shot performance It is found that CLIP’s zero-shot performance is still quite weak on several kinds of tasks. When compared to task-specific models, the performance of CLIP is poor on several types of fine-grained classification such as differentiating models of cars, species of flowers, and variants of aircraft. CLIP also struggles with more abstract and systematic tasks such as counting the number of objects in an image. Finally, for novel tasks which are unlikely to be included in CLIP’s pre-training dataset, such as classifying the distance to the nearest car in a photo, CLIP’s performance can be near random. Challenges in understanding the images that are out of the training set (distribution) While zero-shot CLIP generalizes well to many natural image distributions as investigated, the researchers observed that zero-shot CLIP still generalizes poorly to data that is truly out-of-distribution for it. For example, CLIP learns a high-quality semantic OCR representation that performs well on digitally rendered text, which is common in its pre-training dataset, as evidenced by performance on Rendered SST2. However, CLIP only achieves 88% accuracy on the handwritten digits of MNIST. An embarrassingly simple baseline of logistic regression on raw pixels outperforms zero-shot CLIP. Both semantic and near-duplicate nearest-neighbour retrieval verifies that there are almost no images that resemble MNIST digits in our pre-training dataset. This suggests CLIP does little to address the underlying problem of brittle generalization of deep learning models. Instead, CLIP tries to circumvent the problem and hopes that by training on such a large and varied dataset, all data will be effectively in-distribution. This is a naive assumption that, as MNIST demonstrates, is easy to violate. Unable to generate captions Although CLIP can flexibly generate zero-shot classifiers for a wide variety of tasks and datasets, CLIP is still limited to choosing from only those concepts in a given zero-shot classifier. This is a significant restriction compared to a truly flexible approach like image captioning which could generate novel outputs. CLIP does not address the poor data efficiency of deep learning CLIP also does not address the poor data efficiency of deep learning. Instead, CLIP compensates by using a source of supervision that can be scaled to hundreds of millions of training examples. If every image seen during training of a CLIP model was presented at a rate of one per second, it would take 405 years to iterate through the 12.8 billion images seen over 32 training epochs. Combining CLIP with self-supervision and self-training methods is a promising direction given their demonstrated ability to improve data efficiency over standard supervised learning. Realtime Applications Some of the areas where CLIP has been used for solving real-time use cases are: There is a website called “paint.wtf” where we can play Pictionary. The players here will be judged by the CLIP. CLIP can be used for implementing filters such as “NSFW  (Not safe for work)”. “DALL-E”, an algorithm by Open AI is using CLIP as a helper model as we discussed earlier. CLIP is used to index photos on websites like . Unsplash CLIP can be used to find appropriate images for complex languages such as poetry, riddle, rhymes, novels etc. CLIP can also be used to pick images that are corrupt or distorted. A new research paper titled, ‘ ,’ demonstrates how a supervised inversion method was used to get effective representations of corrupt images. Inverse Problems Leveraging Pre-trained Contrastive Representations Released in 2021, a generative model called CLIP+VQGAN or Vector Quantized Generative Adversarial Network is used within the text-to-image paradigm to generate images of variable sizes, given a set of text prompts. However, unlike VQGAN, CLIP isn’t a generative model and is simply trained to represent both images and text effectively. Conclusion It is an undeniable fact in the deep learning industry that CLIP has paved a way for the development of lof advanced algorithms for solving complex use cases related to image processing and NLP. CLIP can be considered an innovative aqueduct between computer vision and NLP. Also, Since it does not require task-specific training data, it is possible to feed huge amounts of text data and it would slowly get better and better at more unrelated tasks. We can together eagerly wait for the breakthrough advancements that CLIP will be provided in the future. I hope you have got a basic introduction to the concept behind the CLIP algorithm in a lucid manner. I have added the links to the research paper in the reference section which you can use in case you need to refer to the in-depth implementation. References CLIP documentation from Open AI - Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever “Learning Transferable Visual Models From Natural Language Supervision" - Sriram Ravula*, Georgios Smyrnis*, Matt Jordan, Alexandros G. Dimakis, The University of Texas at Austin, NeurIPS 2021 “Inverse Problems Leveraging Pre-trained Contrastive Representations" Katherine Crowson, Stella Biderman,, Daniel Kornis, Dashiell Stander, Eric Hallahan, Louis Castricato, and Edward Raff "VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance"- Radford, Alec; Narasimhan, Karthik; Salimans, Tim; Sutskever, Ilya (11 June 2018). (PDF). . p. 12. (PDF) from the original on 26 January 2021. Retrieved 23 January 2021. "Improving Language Understanding by Generative Pre-Training" OpenAI Archived Johnson, Khari (5 January 2021). . VentureBeat. from the original on 5 January 2021. Retrieved 5 January 2021. "OpenAI debuts DALL-E for generating images from text" Archived Ramesh, Aditya; Dhariwal, Prafulla; Nichol, Alex; Chu, Casey; Chen, Mark (12 April 2022). . : . "Hierarchical Text-Conditional Image Generation with CLIP Latents" arXiv 2204.06125

CLIP: An Innovative Aqueduct Between Computer Vision and NLP

CLIP: un acueducto innovador entre la visión artificial y la PNL

CLIP : un aqueduc innovant entre la vision par ordinateur et le TAL

क्लिप: कंप्यूटर विजन और एनएलपी के बीच एक अभिनव एक्वाडक्ट

CLIP: コンピュータ ビジョンと NLP の間の革新的な水路

CLIP: um aqueduto inovador entre visão computacional e PNL

CLIP: Một cống dẫn nước sáng tạo giữa thị giác máy tính và NLP

CLIP：计算机视觉和 NLP 之间的创新渡槽

If you want to know more about printing labels using TSPL commands, then you can check my previous articles: How to Print labels with TSPL and JavaScript How to Print images with TSPL and JavaScript But we can also print receipts using the same TSPL commands and use the same printer as a receipt and label printer. Some of our customers from “ ” asked us if they can use their label printer as a receipt printer. So, we implemented this feature, and I wanted to share what should be considered while switching between these two modes. Alto’s POS & Inventory Height of the receipt We use label rolls to print labels and continuous paper rolls for receipts. In the beginning, we can set the size and print multiple labels. This won’t be an issue because every label at the label roll has the same size. But the situation is different when printing receipts because every receipt can have a different height. Generally, receipt printers use language like ESC/POS. One of the differences between TSPL and ESC/POS is we need to set the size of the receipt while printing. So when we build a receipt with the commands like , , and we need to calculate the height of the receipt according to the items that we have. TEXT BARCODE BITMAP We manually specify the height of the items like barcode and bitmap so we already know their height, but we should calculate the text height according to the font that we use. Let’s see what the text command looks like: TEXT x,y,“font”,rotation,x-multiplication,y-multiplication,[alignment,] “content” Every “font” has a different size: 1: 8 x 12 fixed pitch dot font 2: 12 x 20 fixed pitch dot font 3: 16 x 24 fixed pitch dot font 4: 24 x 32 fixed pitch dot font 5: 32 x 48 dot fixed pitch font 6: 14 x 19 dot fixed pitch font OCR-B 7: 21 x 27 dot fixed pitch font OCR-B 8: 14 x25 dot fixed pitch font OCR-A So if the receipt contains a single text like this: TEXT 10,10,"2",0,1,1,"FONT 2" Then the receipt height should be: y + [font height]*[y-multiplication] 10 + 20*1 = 30dots Reset the printer Generally, the printer takes the size at the beginning and will use that size for all labels until we manually reset the printer. In another word, if we print a receipt with a height of 100 and then if, we try to print a new receipt with a height of 50, it will continue printing with the height of 100. To prevent this, we can reset the printer with the command , and it will use our new size: <ESC>!R <ESC>!R
SIZE 48 mm,25 mm
CLS
TEXT 10,10,"4",0,1,1,"HackerNoon"
BARCODE 10,60,"128",90,1,0,2,2,"altospos.com"
PRINT 1
END Printing with JavaScript (Node.js) I created a simple example that prints two receipts with different heights. The first receipt’s height is about 90 dots (including paddings), and with the 203 DPI printer, it will be around 11 mm ( ). And the second receipt’s height is 21 mm. If we do not reset the printer ( ), the second one will also have a height of 11 mm and the remaining part won’t be printed. learn more <ESC>!R const usb = require('usb');

(async function() {
  await print([
    '<ESC>!R',
    'SIZE 80 mm,11 mm',
    'CLS',
    'TEXT 10,10,"4",0,1,1,"HackerNoon"',
    'TEXT 10,47,"1",0,1,1,"Top Stories"',
    'TEXT 10,64,"1",0,1,1,"Programming"',
    'PRINT 1',
    'END',
  ]);

  await print([
    '<ESC>!R',
    'SIZE 80 mm,21 mm',
    'CLS',
    `TEXT 10,10,"4",0,1,1,"Alto's POS & Inventory"`,
    'TEXT 10,50,"2",0,1,1,"Restaurants & Cafes"',
    'TEXT 10,75,"2",0,1,1,"Retail Stores"',
    'TEXT 10,100,"2",0,1,1,"Services"',
    'TEXT 10,125,"1",0,1,1,"https://altospos.com"',
    'PRINT 1',
    'END',
  ]);
})();

function print(cmds) {
  return new Promise(resolve => {
    // you can get all available devices with usb.getDeviceList()
    let device = usb.findByIds(/*vid*/8137, /*pid*/8214);
    device.open();
    const interface = device.interfaces[0];
    interface.claim();
    const outEndpoint = interface.endpoints.find(e => e.direction === 'out');
    outEndpoint.transferType = 2;
    outEndpoint.transfer(Buffer.from(cmds.join('\r\n')), (err) => {
      device.close();
      resolve();
    });
  });
} I used the ‘ ’ package to connect and send the commands (On Windows, you may need to install a driver, ). usb to learn more visit the package’s page No War! ✋🏽

How to Print Receipts With TSPL and JavaScript

Artificial Intelligence is currently one of the most sought-after technologies, and so is NLP, a subset of AI. Recent years have been marked by tremendous research and development in AI. NLP is an important technology, and we use it every day. This article will give us some perspective on this emerging technology. We will look at its history and some of its typical applications. How and where can we learn NLP to create various real-life projects. We also briefly look at Python, arguably the most favorable programming language for NLP and some of its essential NLP libraries. Along the way, some learning resources have also been recommended. What is NLP? First things first: what exactly is NLP? We all know that computers can only understand 0s and 1s, but we humans communicate in various languages, thousands of languages. , to be exact. This creates a deadlock in our interaction with the computers. The traditional solution to this deadlock was a methodical interaction with computers by clicking on various options to perform the desired task. But nowadays, we can achieve this by talking to our computers through Cortana or Siri in our own language, and they can serve our tasks and even talk back to us in the same language. The underlying technology that makes this happen is Natural Language Processing or the NLP. 7139 NLP is an emerging interdisciplinary field of linguistics, computer science and artificial intelligence that makes computers understand and generate human language. This is done by feeding a large amount of data or corpus in case of language data to the computer, which analyses them to understand and generate meanings. History of NLP Although NLP has become very popular in recent years, its history goes back to the 1950s. The genesis of all the AI technologies lies with the computer scientist Alan M. Turing and his seminal paper titled .” The central question of his article was ‘ ’. In the report, Mr Turing proposed a benchmark to test the intelligence of computers in comparison to humans. He called it the Imitation Game, which later became known as the . Among other criteria was the machine’s ability to ‘understand and speak’ natural languages. The of 1954 was another development early in machine translation. The experiment demonstrated a fully automatic translation from Russian to English. The field of NLP, along with AI, has undergone continuous development since then. The growth in the field of NLP can be divided into three categories based on the underlying method or approach for solving NLP problems described in the next section. “ Computing Machinery and Intelligence (1950) Can machines think? Turing Test Georgetown experiment NLP Methods Rule based NLP (1950s — 1990s) The earliest methodologies used to achieve natural language processing by computers was based on a pre-defined set of rules, also called Symbolic NLP. A collection of hand-coded rules was fed into the computer, and it yielded results based on that. The early research in NLP was focused primarily on machine translation. Rule-based machine translation (RBMT) required a thorough linguistic description of the source and the target languages. The basic approach involved two steps: 1) finding a structural equivalent of the source sentence and the output using a parser and an analyzer for the source language and a generator for the target language, and 2) using a bilingual dictionary for a word-to-word translation to finally produce the output sentence. Given that a human language is pervasive and ambiguous, infinitely many such rules are possible. Obviously, hand-coding such a large number of rules is not possible. Thus, these systems were very narrow and could only produce results in some given scenarios. For example, the much-celebrated Georgetown experiment could only translate some 60 plus sentences from Russian to English. Statistical NLP (1990s — 2010s) With the advent of more powerful computers with higher processing speed, it was now possible to process a large amount of data. Statistical NLP took advantage of this, and new algorithms based on machine learning came into being. These algorithms are based on statistical models which make a soft, probabilistic decision to yield output(s). Instead of hard-coding rules as in Rule-based NLP, these systems automatically learn such rules by analyzing large amounts of data fed through real-world parallel corpora. In the case of Statistical NLP, the system does not generate one final output. Instead, it outputs several possible answers with relative probability. The drawback, however, was that its algorithms were challenging to build. It required a complex pipeline of separate sub-tasks like tokenization, parts-of-speech tagging, word sense disambiguation, and many more to finally produce the output. Neural NLP (2010s — present) As deep learning became popular during the 2010s, the same was applied to the NLP. Deep neural network-style machine learning methods became widespread in NLP too. This approach, too, uses statistical models to predict the likelihood of an output. However, unlike Statistical NLP, it incorporates the entire sentence in a single integrated model wiping out the need to build that complex pipeline of intermediate subtasks as in statistical models. In this approach, the system uses an artificial neural network. The artificial neural network system, in theory, tries to mimic the neural network of the human brain. An artificial neural network consists of three layers: input, hidden, and output. The input layer receives the input and sends them to the hidden layer, where all the computations are performed on the data. The output is then transferred to the output layer. The connection between the neurons is called weight. The initial value of weights is set randomly, which changes during artificial neural network learning. These weights are crucial in determining the probability of an output. Common NLP Tasks Here is a non-exhaustive list of some of the most common tasks in natural language processing. Note that some of these tasks may not be an end in themselves but serve as subtasks in solving other tasks which have real-world applications. — separate a continuous text into words Tokenisation — determine the parts of speech of each word in a sentence Parts-of-speech tagging — filter out high-frequency words like to, at, the, for, etc Stopword removal — remove inflections (prefix and suffix) (e.g., driving → driv) Stemming — remove inflections and return base form of the word (e.g., driving → drive) Lemmatization — determine which word refers to which words in a sentence/text Coreference resolution — determine and visualize the parse tree of a sentence Parsing — select contextual meaning of a polysemic word Word sense disambiguation — determine the proper nouns in a sentence/text Named entity recognition — identify relationships among named entities Relationship extraction — determine the text printed in an image Optical character recognition (OCR) — convert speech into text Speech Recognition — separate a speech into words Speech Segmentation — convert text to speech Text-to-speech — produce a summary of a larger text Automatic summarisation — detect and correct grammatical errors in a text Grammatical error correction — automatically translate a text from one language to another Machine translation — convert text into machine-readable code Natural language understanding (NLU) — make machine produce natural language Natural language generation (NLG) Python for NLP Python is a preferred programming language for NLP. As of February 2022, it is the programming language. Python’s ubiquitous nature and its application to a wide array of fields make it so popular. most popular While programming languages like Java and R are also used for NLP, Python is a clear winner. Python is easy to learn and understand because of its transparent and straightforward syntax. Python offers perhaps the largest community of developers, which can be really helpful in case the code needs some debugging. In addition, Python seamlessly integrates with other programming languages and tools. Most importantly, Python is backed by an extensive collection of libraries that enables developers to quickly solve NLP tasks. Resources : Start with this five-part specialization program on Coursera. It will provide a complete overview of Python programming. Python for Everybody : Read this free to read online book by Al Sweigart for step-by-step instructions and guided projects. Automate Boring Stuff with Python the official Python tutorial and documentation : Python tutorial NLTK NLTK — the Natural Language Toolkit — is a suite of open-source Python modules, data sets, and tutorials supporting research and development in Natural Language Processing. is the most popular library for NLP. It has a massive active community with over 10.4k stars and 2.5k forks on its . It was developed at the University of Pennsylvania by Steven Bird and Edward Loper and was released in 2001. NLTK is freely available for Windows, Mac OS X, and Linux. It has built-in support for more than 100 corpora and trained models. NLTK comes with a free written by its creators, a comprehensive guide for writing Python programs, and working with NLTK. They also have an active on google groups. Natural Language Toolkit or NLTK GitHub repository ebook discussion forum Resources: : Watch this introductory video on freeCodeCamp’s YouTube channel for getting started Natural Language Processing (NLP) Tutorial with Python & NLTK : Read the official NLTK book written by its creators for a deeper understanding of NLP and NLTK Natural Language Processing with Python spaCy Industrial-strength Natural Language Processing (NLP) in Python. is relatively young but very hot right now. Its has more than 22.4k stars and 3.7k forks, much higher than NLTK. It was written in Python and Cython, making it fast and efficient at handling large corpus. It is an industry-ready library and was designed for production usage. spaCy GitHub repository Some features of spaCy are: It provides support for linguistically-motivated tokenization in more than 60 languages. It has 64 pre-trained pipelines in 19 different languages. It provides pretrained transformers like BERT. It provides functionalities for named entity recognition, part-of-speech-tagging, dependency parsing, sentence segmentation, text classification, lemmatization, morphological analysis, entity linking and more. Resources: : freeCodeCamp’s video tutorial for getting started Natural Language Processing with spaCy & Python — Course for Beginners : The most essential concepts, explained in simple terms spaCy 101 CoreNLP A Java suite of core NLP tools. was initially written in Java and was developed at Stanford University. But it is equipped with wrappers for other languages like Python, R, JavaScript, etc. Thus, it is a library that can be used with most programming languages. It is a one-stop-shop destination for all the core NLP functionalities like linguistic annotations for text, including token and sentence boundaries, parts of speech, named entities, numeric and time values, dependency and constituency parses, coreference, sentiment, quote attributions, and relations. CoreNLP currently supports 8 languages: Arabic, Chinese, English, French, German, Hungarian, Italian and Spanish. It has 8.3k stars and 2.6k forks on its . CoreNLP GitHub repository Resources: : Official CoreNLP documentation and pipelines Documentation TextBlob Simplified Text Processing is a Python library for processing textual data. It provides a simple API for the most common NLP tasks like POS tagging, tokenization, n-grams, etc. It is beginner-friendly and the fastest among other libraries. and is made on top of and . It has 8k stars and 1.1k forks on its at the time of writing this article. TextBlob NLTK pattern GitHub repository Resources: : Official TextBlob documentation and quick start guide Documentation Gensim topic modelling, document indexing similarity retrieval Gensim is a Python library for and with large corpora. was created by Radim Řehůřek in 2009. It is implemented in Python and Cython, making it incredibly fast. Plus, all its algorithms are memory-independent, i.e. they can process inputs larger than RAM size. It is mainly used to identify semantic similarities between two documents through vector space and topic modeling. It supports algorithms like Latent Semantic Analysis (LSA/LSI/SVD), Latent Dirichlet Allocation (LDA), Random Projections (RP), Hierarchical Dirichlet Process (HDP) or word2vec deep learning. It works on vast data collections from specific categories and provides clear insight. Gensim’s has 12.9k stars and 4.2k forks. Gensim GitHub repository Resources: : Official documentation and tutorials. Documentation Conclusion Natural language processing is a sub-field of Artificial Intelligence under active research and development. We can see its practical applications all around us. From automatic captioning in YouTube videos to Chrome automatically translation webpages in foreign languages for us, assistive writing with Grammarly, our iPhone’s keyboard predicting keywords for un and so on. The possibilities are limitless. Natural language processing is indispensable for artificial intelligence and for our future technologies. References Bird, Steven, et al. . 1st ed, O’Reilly, 2009. Natural Language Processing with Python Budditha Hettige. . 2011. , . A COMPUTATIONAL GRAMMAR OF SINHALA FOR ENGLISH-SINHALA MACHINE TRANSLATION DOI.org (Datacite) https://doi.org/10.13140/RG.2.1.2330.6968 Heller, Michael. ‘Study Claims Siri and Google Assistant Are Equal’. , . Accessed 8 Feb. 2022. Phone Arena https://www.phonearena.com/news/Study-claims-Siri-and-Google-Assistant-are-equal_id115667 Singla, Karan. . 2015. , . Methods for Leveraging Lexical Information in SMT DOI.org (Datacite) https://doi.org/10.13140/RG.2.1.2138.7367 Turing, A. M. ‘I. — COMPUTING MACHINERY AND INTELLIGENCE’. , vol. LIX, no. 236, Oct. 1950, pp. 433–60. , . Mind DOI.org (Crossref) https://doi.org/10.1093/mind/LIX.236.433

Natural Language Processing with Python: A Detailed Overview

Free Software Versus “Open Source” In 1980, a brand new Xerox 9700 printer was installed in an office of MIT’s computer science department. It often jammed. Several people in that department probably could have fixed it if not for the fact that it ran on proprietary Xerox software. One of those people, Richard Stallman, tried to at least program it to notify the rest of his office when it jammed, but wasn’t allowed to do that either. Whenever he and his colleagues needed a printout, they had to go check if it was jammed first. It was on a different floor. Stallman begged a colleague with Xerox connections to share the code for the driver. The colleague, despite having access, was contractually obligated to say no. This experience broke something in Stallman’s brain. It was a catalyst that solidified his resolve to give free software a permanent place in the world. If he couldn’t make all software free, he could at least lay the groundwork for free alternatives. He had the skills to help the world preserve the possibility of fixing your own printer, or asking a nerdy neighbor to do so, rather than remaining at Xerox’s mercy. He would not tolerate a society where people were actively discouraged from helping each other by the fact that Xerox’s profits might suffer a little. He soon left MIT to work on GNU, which stands for “GNU’s Not Unix.” Unix was an operating system built and owned by AT&T. Stallman’s goal with GNU was to build a completely free alternative that could run on any computer. Crucially, when he did this, he created a new type of copyright license to protect it called the GNU General Public License. It’s also called a “copyleft” license because it reverses what a traditional copyright license does: the GPL states not just that anyone can modify or distribute the software, but that any modified versions fall under the same rules. Corporations who tried to build on top of GNU, for example, would have to be careful in how they did so lest their product become free software. His vision was no pipe dream. In the early 1990s, a Finnish student named Linus Torvalds used what already existed of GNU to finish a central piece of the operating system, known as the kernel. The GNU/Linux operating system was born. Stallman, Torvalds, and their globally distributed team showed the world that wildly successful software could be developed both remotely and without corporate funding. Many tools that developers take for granted today, such as the version control system Git, have been developed for this project. Likewise, many huge companies owe much of their profit and success to it. GNU/Linux got so good that countless software giants began using it extensively, such as Google, Apple, Facebook, Amazon, and even Microsoft, despite Steve Ballmer once calling the GPL a “cancer.” (They’ve been careful, of course, to not do so in a way that makes their products free.) That such a project could be built this way feels obvious now, but it wasn’t obvious at all back then. In 1997 a hacker named Eric Raymond, inspired by his experiences working on GNU/Linux, published an called “The Cathedral and the Bazaar,” arguing that building software in this open, communal, bottom-up way, rather than the private and top-down way favored by most companies, yields inherently better results. Putting vastly more eyeballs on a project’s code, as it turns out, makes it vastly harder for bugs and security flaws to take root. It doesn’t hurt when a lot of the contributors work on it out of pure interest and fun, either, rather than economic need. essay A year later, a browser company called Netscape did something crazy. Desperate to stop Microsoft’s Internet Explorer from cutting into their market share and inspired by this essay, they publicly shared the code for their latest web browser, inviting the world to build it with them. In the same initiative, with help from supportive hackers like Raymond himself, they also coined the term “open source.” The nascent open source community ended up , but the move heralded the future. Companies were now allowed, even encouraged, to start projects where large chunks of development, along with distribution and branding, still happened in-house, but where the whole world could contribute. (Projects like React continue to see great success under this model.) From their perspective, it makes perfect sense: if people are going to build free software anyway, why not benefit from that? scrapping the browser and starting Mozilla.org instead In giving companies a free pass to enter the “open source community,” however, certain hackers said “take what you want and give what you want” to a bunch of organizations built around maximizing the ratio of the former to the latter. Here we are with Schrödinger’s open source, both broken and unbroken. It’s unbroken in the sense that people still write terrific software because they can and want to, and the world still benefits from it. There is also, of course, a darker side. The recent is merely the latest dancer in a long conga line of problems stemming from the unholy marriage of the cathedral and the bazaar. Companies adapt an open source library without contributing; the library falls into disrepair because its maintainers can’t work for free forever; security vulnerabilities arise that the companies, building their cathedrals, don’t notice or address until irreparable damage has been done. log4j fiasco These companies often have the nerve — as Microsoft did in 1998 in response to Netscape’s “stunt” — to claim that the open source maintainers are the ones being risky, and that the cure is pouring even more trust and resources into their company, which know what it’s doing. Again, though, they’re incentivized to be this way. Software consumers like you and me, instead of rallying around better free software, want and use their products. The collective demand that this puts on good cheap software has direct impacts on the maintainers of the free software underlying it. Facing increasing pressure without proportionate rewards, maintainers can burn out. Even Guido van Rossum, “benevolent dictator for life” of Python, . Linus Torvalds from Linux. Jacob Thornton, whose on the history of open source I’ve been paraphrasing large chunks of, references “cute puppy dog” syndrome: you start a project because it’s fun, but then it grows. It becomes a more thankless job. While it’s hard to pass off your baby to people who won’t care for it like you did, it’s often harder to take care of Clifford the Big Red Dog when everyone wants to play with him and no one wants to finance his kibble. stepped down took a break talk Filippo Valsorda, an engineer on the Go team at Google with extensive open source experience, forcing companies to “professionalize” (invoice without hiring) the maintainers of the open source projects they use. I suspect this suggestion will grow in popularity; it lets companies part with some extra cash rather than their fundamental assumptions about how freely shareable software should be, or about who should have input in decisions about software that affects the entire world. (Some have carried the torch of questioning these assumptions, like the late Aaron Swartz in his unfinished work about the , though they’re few and far between.) recently suggested Programmable Web In a 2001 where he told the Xerox printer story, Stallman noted that in the 1970s, the heady days of free software being the norm, none of this was an issue. The survival of your project didn’t depend on funding or software licenses. It didn’t really even depend on what your project had. People contributed work they cared about. They debated it carefully. They delegated decisions about specifics to people who clearly knew and cared about those specifics, probably because they’d already worked on them. There was also no insane reactionary erasure of all governance whatsoever, as some have found in “flat” organizations; there was no leaving people to grapple with the — informal cliques of power — and no driving people to because they cared more than the higher-ups knew how to handle. (Or because they kept running into buggy proprietary software they weren’t allowed to fix.) speech model of governance Tyranny of Structurelessness burn out and disengage It was all a “do”-ocracy. People weren’t in it for power and clout; they were in it for fun and community. “Cooperation was our way of life,” Stallman said. The Indigenous Critique For a long time after the Renaissance, European culture actually stayed pretty authoritarian, hierarchical, and bound by strict puritanical codes of morality. People like Jean-Jacques Rousseau who finally started touting “Enlightenment” values — individual liberty, questioning authority, rational debate, all that good stuff — often did so with traceable influence, sometimes even explicitly-stated influence, from Native Americans, many of whose nuanced and well-reasoned critiques of the West (read up on for a good example) found their way into widely-read books written by New World explorers. Kondiaronk In fact, it was natives of the American northeast and Great Lakes regions who were already living these values. They prized and prioritized personal autonomy, doing virtually nothing they morally disagreed with, to an extent that would make any modern libertarian jealous. They went so far with those principles as to automatically ensure a social safety net. When you’re going hungry or otherwise in mortal danger, you don’t have much personal autonomy, as you’re just trying to survive. The community would help you out. One could say the reverse was true; in caring for each other, they protected personal autonomy. It doesn’t really matter. This was just how they lived. Cooperation was their way of life. There was no blind, reflexive, dogmatic reverence to the authority of the humans you lived with beyond their ability to morally and logically convince you. Words like “lord” and “commandment” were hard for early missionaries to translate into the natives’ languages. Rigorous notions of private ownership were not really a thing; in practice, these amount to telling everyone else in your community “you cannot access this, even if you find it helpful; it’s mine.” Amassing power over others wasn’t really a thing either. The mechanisms that allowed it in European society, like money and social status, were simply not that important, even if they existed. The European reflex to sanctify and protect ideas like these was alien, especially when it came at the cost of actual human life. This had the interesting effect of translating directly into the kind of “equality under the law” that the Founding Fathers began enshrining. (If anything, the Greco-Roman traditions that my high school history books instead pointed to when explaining the Founders’ talk of “equality” merely kept it limited to property-owning white males.) The natives, in living this way, flouted the European cultural assumption that individual liberty and social cohesiveness were at odds. It was literally revolutionary. In the centuries since, the tendency of Western historians and anthropologists, lulled by stereotypes of the “noble savage” (or simply the “savage”), has been to discount the direct shoutouts of people like Rousseau, as though they couldn’t be accurate. These thinkers must have been trying to seem “exotic,” or were just joshing us, right? Indigenous cultures were too “simple,” “innocent,” and “primitive” to know what they were talking about when it came to statecraft, right? possibly The time has come to show my hand again. Much of what you’ve just read is a direct paraphrase of the beginning of the book by David Wengrow and David Graeber. They go on: The Dawn of Everything What if the sort of people we like to imagine as simple and innocent are free of rulers, governments, bureaucracies, ruling classes and the like, not because they are lacking in imagination, but because they're actually more imaginative than we are? We find it difficult to picture what a truly free society would be like; perhaps they have no similar trouble picturing what arbitrary power and domination would be like. Perhaps they can not only imagine it, but consciously arrange their society in such a way as to avoid it. The historical record shows that even the Europeans preferred these ways. Benjamin Franklin himself wrote puzzled and disgruntled accounts of how pretty much anyone with years of experience living among both indigenous people and Europeans (via adoption, kidnapping, etc.) would overwhelmingly end up choosing to live amongst the Native Americans. As Wengrow and Graeber write: “There is the security of knowing one has a statistically smaller chance of getting shot with an arrow. And then there’s the security of knowing that there are people in the world who will care deeply if one is.” Decriminalizing Caring The type of world you live in comes from the way your society values and prioritizes different freedoms. Everyone, by nature, is “free” to do anything, including murder; when we deny the freedom to murder, though, it protects our freedom to do much more. So while freedom is not always a zero-sum game, like when people write useful free software that enables greater autonomy for all, there can be tradeoffs. The current trouble with open source comes from the fanatical worship and defense of one particular freedom at the exclusion of others. It is the freedom to own and hoard property — to keep software (or land, or whatever else) private and maximally-profitable by all means necessary — even at the cost of our freedom to help each other, breathe clean air, enact a 4-day workweek that lets us write more free software, or not be by Chevron. wrongfully imprisoned As those examples show, the freedom to hoard is tied up in a vicious cycle (well, virtuous for some) with the freedom to use money to influence laws. People from a certain class, like Billy McFarland of Fyre Festival infamy, get 6 years in prison for committing tens of millions of dollars in wire fraud, and that’s when they get caught and publicized; poor people routinely go to jail for life for . stealing $50 of food As Stallman mentions in his 2001 speech, when something (like a computer, or a justice system) is broken, people stop caring as a defense mechanism: You know if the computer is constantly frustrating to use, and people are using it, their lives are going to be frustrating, and if they're using it in their jobs, their jobs are going to be frustrating; they're going to hate their jobs. And you know, people protect themselves from frustration by deciding not to care. So you end up with people whose attitude is, "Well, I showed up for work today. That's all I have to do. If I can't make progress, that's not my problem; that's the boss's problem." And when this happens, it's bad for those people, and it's bad for society as a whole. The status quo is that workers in many industries, not just open source software, are increasingly overburdened and undercompensated, largely due to the same dynamics that hobble the justice system. Companies take, and fail to give, as much as the law will allow, and “money as speech” means the law will allow a lot. The minimum wage, especially compared to inflation, remains a sad joke. In tandem with spending less time caring about and helping each other, then, we therefore have to spend more time grinding, hustling, and seeing each other as competition, which compounds the problem. Our financial rewards for doing so are artificially suppressed by both corporate lobbying and government unresponsiveness (but I repeat myself). Charlie Warzel makes this connection in a recent though unfortunately paywalled called . There is a growing realization — not just among software developers, but among Kellogg’s workers and healthcare professionals and others — that we need to force necessary changes in how companies not only pay the workers, but also invest in the systems, that sustain them. article Guardians of the Internet Our world obviously depends on systems: software systems, energy systems, healthcare systems, and more. The legwork of keeping these systems going falls to workers. We have to incentivize caring for those people more than the brand names that leech off of them, lest those people, and those systems, fall apart. I urge you to ask yourself, not just once but often, if it’s really worth prizing private property and its protection as we do (i.e. above the protection of all else) or instead if, as the natives did, we should prize community and personal autonomy, perhaps even more — crazy, I know — than private property. Ask if you want your laws to enshrine the freedom to care only and solely for yourself, or the freedom to cooperate. As the early free software community shows, living in a complex technological world doesn’t mean the latter is blocked. It can be harder, of course, given all the corporate resistance one may face, but it’s possible. Since I value autonomy, I’ll respect your choice; I just find it a pretty obvious one. Previously published here.

Cooperation, Survival, Open Source, and Decriminalizing Caring 

A comprehensive work on how the biggest names in Identity Verification industry were tricked, trolled and fooled by a simple method. I never imagined that one day I would conduct fake identity verification demos in the name of investigative journalism online. But that’s what we did and as it came out it’s not too easy to secure online marketplaces. I thought that ID verification is foolproof, but clearly, it’s not. Identity verification has been the talk of the town ever since terrorist funding and terrorism increased in the world, so we decided to see how reliable these services are. Identity verification — The ultimate security and compliance solution for digital platforms? In this digital age, identity theft, cyber crimes, and ID fraud are a commonly occurring phenomenon in the digital landscape and it requires a permanent deterrence against such issues. Since more business transactions are occurring online. Customers don’t have to be physically present to conduct business, creating many opportunities as well as challenges for businesses. One of the major challenges online businesses face is the verification of customer identity. Several verification services provide expert solutions when it comes to identity-, document-, and or other types of verification. Businesses verify user identities to create trust online and offline, prevent identity fraud, and comply with privacy and anti-fraud regulations. Identity verification can be accomplished through different means such as photo-based identity document verification, face authentication, and checking against the public databases. We tried our luck with the most advanced technologies available today. A tabular representation of the services provided by the top Identity verification service providers: We compiled a list of top identity verification solution providers and conducted extensive testing on their platform to check if businesses can trust these services providers to perform a crucial task of ID verification on their customers. After contacting these services and performing ID verification on their platforms we divided these providers into three subsections and ranked them on the scale of 0–5 depending on parameters like: Pricing, duration of verification, authenticity of verification and customer service. The demo video is available for proof. The identity of the person is hidden due to security and privacy concerns. These rankings are based on parameters devised by us and do not represent or imply anything. Disclaimer: Unexpected names verified the mimicked documents To our surprise, some service providers simply verified the fake documents and declared them original. Here are those providers: 1. Jumio 2.5/5* Ranking: uses the power of AI, biometrics, machine learning, and liveness detection to stop fraudsters from infiltrating your online ecosystem and get in compliance with KYC/AML. Even though Jumio has its fair share of legal trouble and bankruptcy along with the mass clearing of employees in India. It still is regarded as one of the top services Jumio Major Industries Financial Services, Retail, Travel, sharing economy, Gaming, Telcos, Mobility & Healthcare. Countries 200+ Demonstration It was quite hefty to get a demo from Jumio. They interrogate prospects deeply by asking different sorts of verification documents which is quite good because if you are looking to establish business relations with someone you need to ensure everything but the process was very much like a criminal interrogation. Anyhow their verification process is required to be done on different devices. After downloading the software on the phone the results have to be updated on the link provided by them. The process of sending selfies was slow and results were displayed after . When the data was finally uploaded, they verified the fake ID card which was quite interesting because Jumio is regarded as the top verification firm in the sector. 5 minutes Jumio demo — Showing the result of a ‘verified fake’ ID card 2. Cynopsis Solution Ranking: 3/5* automates your client on-boarding and screening processes and conducts ongoing due diligence and record-keeping. Theirs is a product based on screening against AML lists. Cynopsis solution Major Industries Financial Services Sector (including FinTech start-ups) and the Professional Services Sector Demonstration They claim to be the oldest Asian firm. However, their system is easy to surpass. We used a $2 dollar proxy and free online verification number to get through their system. There was no other language found apart from English and the system had no other option. For a country that doesn’t have the option of expiry in their document, they didn’t have the option to deselect it. We had to put a fake expiry date there and the verification process took . Ironically the camera was not getting the liveness detection of a real person so we had to get a studio light to clear the liveness process and get it done twice. With all these hurdles and strict process we never thought they will authenticate fake ID documents but that’s clearly what they did. 15 minutes Cynopsis demo — Showing the result of a ‘verified fake’ ID card 3. ID Scan 3.5/5* Ranking: is enhancing environments and experiences through identity verification and information gathering to improve compliance, safety, and customer satisfaction. ID Scan Major Industries Healthcare, Automotive, Dispensaries, Education, Finance, Gaming, Property Management, Hospitality, Law Enforcement, Night clubs & bars, Non-profit, Retail. Countries US and Canada only Demonstration While talking to their representative we got a sense that they don’t have any idea about the verification or there was some communication gap because the representative was speaking in a Russian accent and at once, we thought that they might be based in Russia. However, this is purely an assumption based on our experience, after all, America and Canada are a land of diverse people. But they were unable to verify ID from non-American countries. IDScan demo — This service is only available to Canada and the US 4. ID Now 2.5/5* Ranking: They are one of the leading providers for identity verification in Europe. The platform offers complete flexibility across a wide range of KYC services, from fully automated to agent-assisted solutions as well as design solutions. IDnow Major Industries Finance, Insurance, Automotive, Mobility, Telecommunications, eCommerce, Gaming, Digital Contract Management. Countries 193+ Demonstration The story of getting a demonstration from IDnow is quite long and interesting and may need a full report to explain the difficulties. Nonetheless, keeping things simple and short we had to do some real work to get their demo. The main issue, however, is that they also verified the fake ID. We tested their services from our identity documents and as explained earlier they verified our document. “imaginary” ID Now — Showing the result of a ‘verified fake’ ID card Some rejected the fake IDs, still do they satisfy client requirements? 5. Shufti Pro 4/5* Ranking: is the youngest of the names mentioned here. According to them they are gaining the market share rapidly owing to the “ ” document verification. Shufti Pro fast and secure Major Industries Financial Services, Peer to peer economy, Gaming, Crowdfunding, Telecommunication, Travel Countries 200 Demonstration Getting their demo was not instant, but we got an email with details after 15 minutes. We used European origin cards. Their back-office was complicated but we got our first rejection in 60 seconds. Then we tried with a proxy location and different cards. After the result, we got a message . Then to get rid of them we had to verify with one real ID. The average of 3 verification times was 50 seconds. But we were a bit surprised that they didn’t verify any of the fake documents. “Too many declines, seek expert advice” Shufti Pro — Showing the result of a ‘declined fake’ ID card 6. Sum and Substance Ranking: 3/5* is a KYC/AML platform that helps businesses verify IDs through facial biometrics, document screening, and background checks of new customers in compliance with rules & regulations. Sum & Substance Major Industries Fintech, Trading, Gaming, Transportation, Market-places Countries 150+ Demonstration Our call was received by a guy with a Russian accent, whereas we thought there would be a British guy. He was quite helpful however we can’t say the same about their system. This is visible from the result shown in the image below. They declined the card based on the time taken for them which was 13 minutes. However, they failed to mention the issue with the image: was it quality or something else. We found no guidelines regarding the quality of the image and neither did sum and substance provided us with such. Fake documents were passed easily in terms of identity check. 7. Au10tix Ranking: 1/5* is a global provider of security services to environments such as aviation, border control, and sensitive facilities with more than 20 years of experience. Au10Tix Major industries Financial, Marketplaces, Sharing economy, Telecommunication, Crypto-currency Countries 200 + Demonstration In the call, we were told by their team that: . They are completely based on Artificial Intelligence unlike their competitors and equipped with many other which are hard to recall by a journalist but after 5 server failures. “this is the best service” “technical jargon” They finally provided us with the system which was working, well at-least, it took the image, and again the issue was low light. We took the image in sunlight with a benchmark phone and it still was rejected. The result as displayed in the picture shows the data of a completely different person. After seeing the OCR we decided not to waste time on them. Au10tix — Capturing the wrong name from the ‘fake’ ID card 8. Onfido Ranking: 3/5* uses AI-based technology to assess whether a user’s government-issued ID is genuine or fraudulent and then compares it against their facial biometrics. That’s how they onboard customers remotely and securely by providing KYC/AML services, identity verification, and fraud detection. Onfido Major Industries Financial services, Market places and communities, Gaming, Transport, Retail & eCommerce, Healthcare, and Telecommunications Countries 195 Demonstration They responded very quickly in the first attempt. A greek guy asked for more than $20k minimum purchase before the start. When we said that we have low volume he simply muted our queries. We went again this time with a big volume and a different company. We found them racist because the company was Non-American and European. They refused to contact us and we had to get in touch with their senior management for a reply. After that, they fired a barrage of documents and still no demo. They asked us to send them complete details (images, codes of website, etc) which are very risky for a small business owner and a technopreneur. Code can be sabotaged and the lady leading the conversation was discriminating toward the person because the chosen location check was an Arab country. No demo available. 9. Trulioo Ranking: 4/5* helps businesses meet compliance requirements, mitigate fraud risks, and increase trust and safety online through AML watchlist check, ID document verification, Electronic identity verification, Business verification, and Data partners. Trulioo Major Industries Financial Institutions Countries 195 Demonstration They were very good on call and customer support was excellent. They cordially deal with EU companies. Whereas their dealings with Asian clients were far from satisfactory. When we pursued as an Asian, the response was quite disappointing, still, after too many questions asked by them, fellow journalists decided not to pursue after too much documentation requirements. Also, we were asked to deposit a fee before getting a demo. No demo available. 10. Zeal ID Ranking: 2/5* is an electronic signature, it is an in-app registration. Accepted by all public authorities in the EU. And admissible as compliant identification for any licensed bank or financial provider in the EU. Zeal ID Major industries Banks, Financial institutions Countries 25 markets in the EU Demonstration Zeal ID appears as a Google Ad when searching for Trulioo. After getting their demo we encountered a major issue, their link was not working, we got stuck at the first step of facial recognition. So we had to abandon our attempts to get our documents verified. We are sure this might be good in the future but at the moment it seems nothing more than an unfinished launched product. Zeal ID — The server was not working 11. GBG PLC Ranking: 1.5/5* establishes trust between businesses and their customers to help organisations quickly validate and verify the identity and location of customers. GBG Major industries E-commerce, Fintech, Gambling, Public sector, Retail banking Countries 70+ Demonstration They are just another giant who added KYC service in their portfolio, not a product for everyone because there is no product for small and medium buyers just imagine the cost of a custom system. They provide customised solutions as per the requirement of the clients. So, No demo available. Reasons that made us investigate the authenticity of identity verification Jumio Jumio perhaps is the biggest name when it comes to the verification service providers. One would think that they would have covered all the issues. However, it might not be that true as only recently Jumio was called upon a platform saying: A headline of an e-paper following the Jumio scandal This is one of the biggest scandals there can be, it also shows callousness when it comes to handling data especially sensitive data. It is up to you when it comes to data. The company was sued by its own investor in 2016. So a company that has a tricky past can cause big problems for your brand in the future. In the other news: Another headline of an e-paper following the Jumio scandal Jumio had to reach a settlement under a class-action lawsuit as reported by the biometric update. This is shameful as it shows the grey areas of the giant even though they claim to be extremely cautious when it comes to data privacy. Employee security is another thing that matters when you are handing your business to someone with such a sketchy past. Onfido Onfido is a well-known name in the industry and has a reputation to consider, but at this moment that reputation is on the line: A headline of an e-paper following the Onfiido scandal So what is the Informed Consent Act that Onfido has allegedly breached? was passed by the Illinois General Assembly back in 2008, it makes a collection of data without informed consent as an unlawful act, it specifically refers to biometric information. Now if you look at the title it shows an alleged breach of collection of data without consent. Biometric information is personal information so it is a privacy breach also. Onfido looks like they are in deep water and they will have to dig their way out of it. The Biometric Information Privacy Act Concluding remarks This work is based upon our research and how we reached these big guns, your experiences might differ but we have looked into these every possible way. You might not agree with it but you must understand that as an individual work, it is all based on experiences, the intention of this report is not to demean anyone but a mere comparison amongst the big boys in the industry. The End. Disclaimer: This article may contain affiliate links or referral tracking links. We make no warranties of any kind, express or implied, about the completeness, accuracy, reliability, suitability, or availability concerning the directory or the information, products, services, or related graphics contained in the report for any purpose. Any reliance you place on such information is therefore strictly at your own risk. Furthermore, the opinions here belong to the author alone and don't reflect the views of the Hacker Noon organization. Also published on: https://saramonroe17.medium.com/identity-verification-service-providers-the-good-the-bad-and-the-ugly-2ec8ce907550

Ranking and Reviewing the Top Identity Verification Service Providers

Living in the UK, it is almost impossible not to be tempted to try the new challenger banks such as Revolut, Starling and Monzo. I am currently using Revolut as my main bank account and felt the need to access it from my computer. Alas, Revolut does not offer a web app and it does not seem to be in their pipeline, so I went ahead and started building a small web app for my personal use. I quickly found that I could not hit their API in a browser, preflights requests were blocked due to a lack of CORS headers. So to bypass it I started looking at ElectronJS, making the requests using NodeJS and bridging the results using Electron IPC mechanism. It was too cumbersome, so I started checking at another way of making http requests from the renderer side without obeying CORS. One way would be to disable web securities in the chrome flags, but since the app would be to access my bank, it wouldn't be a good idea, so I opted to enable node integration, and used Axios with node's http adapter. It works seamlessly albeit not being able anymore to see requests in the devtools. For security reasons, I wanted to use SSL certificate pinning but it turned out Axios does not provide this option out of the box so I submitted a to add it, this way NodeJS blocks outgoing http requests if the fingerprint of the certificate does not match our pinned fingerprint, preventing proxy eavesdropping and man-in-the-middle attack. PR Later on I faced a common UI problem, since the list of my transactions was large, containing 2000+ items (hey please don't judge :p) React was taking a long time rendering it all at once. I did not want to use react-virtualized, mostly because I had poor experiences using it in the past and that it would break searching in the DOM for text, so I opted to manually delay the rendering of items in the list using a react hook, and to reduce memory footprint and improve scrolling performance, I chose to use the WebComponent since Electron gives us the ability to enable experimental blink flags. <virtual-scroller> The main drawback of using that WebComponent is that list items must be and since I am a big fan of everywhere, I had to style the old fashion way again ... display: block display: flex For the navigation, I decided not to use react-router as it is a rather simple app. Electron does not show URL anyway, so I shouldn't have to deal with routes. Instead, I used a combination of localStorage and React Context API, and was satisfied of the outcome. For animations, I used React-Lottie to have lightweight, high frame rate, JS/SVG animations. Most of the application were made with React using Styled Components. To render the cards, I had to absolute position text elements on a div with rounded corners and pick the same font as credit cards use: OCRA10. The end result is pretty close to what the mobile app displays. I hope you fellow hackers enjoyed the tips and tricks to bypass CORS with Electron, be sure to check out ! the repository

Making an Electron 7 React app for the challenger Bank Revolut

Since the beginning of time, humans have collaborated together to create some of the most incredible civilizations, technology, art and demonstrations of our ability when we work together toward a common goal. Throughout the ages, there have been different methods to facilitate progress, negative and positive, from slavery to sheer passion. Today, we believe value exchange is the key to collaborative efforts and that entire ecosystems can exist with basic protocol rules, yet all other remaining decisions and growth being determined by the user population. My expertise is in recruitment. I have spent my entire career in this industry and have always tried to be the forefront of the technology behind all the various services, from job boards, and matching, to applicant tracking systems that have brought companies and candidates together. But my interest runs much deeper than these logistical technologies. I would say I am a dreamer, a tech evangelist and someone who loves to challenge the status quo. What would an ecosystem designed simply for hiring need to truly be completely self sufficient and ever evolving? Beyond placing people into work, could this ecosystem develop its own trust ranking, enhance and develop its own rules, could it improve its own user interface? Could it even develop itself? I genuinely believe this is possible and I am sure that we are not that far away from it. Earlier this year I was in Hong Kong speaking with a certain level of confidence on the potential role of blockchain technology within recruitment, sure I had all the answers to what the working world would look like in this context. Unexpectedly, I was blindsided by insight from an expert on the panel when they posed the scenario of a working world where people had 5,6 even 8 separate jobs. My instant opinion was, “Impossible”? 8 Jobs? When would people sleep? But the seed was planted and it got me thinking about what he really meant. He was talking about a ‘Gig Economy’, where rather than being centralized to one company, we would all work multiple jobs, offering out our services to different companies and platforms, acquiring incremental levels of pay that would culminate together to create an overall level of income suitable for to maintain a person’s livelihood. Eureka. Blockchain can potentially offer silos of ecosystems based on separate parts of people’s lives and behavior, such as getting a job, getting a ‘Lyft’, getting social and even gaming. Imagine living your life through these separate decentralized ecosystems and that your role was to satisfy any requirements you have and that you could also contribute in micro ways to the upkeep, improvements or tasks for that ecosystem, earning yourself income that could be used in the ecosystem or outside of it in your daily ‘real’ life. When you consider this, we manage ecosystems now to a lesser extent, but they are centralized, like Facebook, Uber, Banking, Shopping, Dating etc… but we received nothing in the form of reward. With the right framework and the right set of starting rules, we could decentralize almost anything and see the revenues that once were controlled by a select few, dispersed amongst the community that has made them so powerful. Blockchain is going to shift the balance of power. So what would this framework look like in the recruitment space? Frameworks for different services would have slight differences, but on the whole, like the laws of physics, all ecosystems that are created would work to the same set of rules and framework. The core constructs of what the framework would need in my opinion are “Rewardocracy” “Trustocracy” “Meritocracy” “Vote-ocracy” I understand that 3 of the words above currently do not exist in the English language, but I am certain you know exactly what I mean and I feel they will become very relevant in the future. Remember that in an ecosystem, the underlying foundation to all the above points is communication, feedback and gamification, none of the above can exist without that. Rewardocracy: Everyone who participates in the ecosystem earns rewards and in the hiring ecosystem that would mean gaining employment, recommending friends to the ecosystem, endorsing skills, endorsing companies, feedback on platform improvements, finding new jobs for the population, improving the technology within it, improving the matching and the list goes on and on. Rewards are not predetermined, a person can request the level of reward they wish to receive from an action and other members of the ecosystem vote as to what is the suitable reward. Similar actions always have guideline reward numbers, but a person can request what they want and the ecosystem can decide what they want to offer, collectively. Trustocracy: The more you are endorsed, the more you contribute and the longer you hold a trust presence within the ecosystem, the greater your trust ranking, that is relative to everything, including the general governance of the ecosystem increases. Trust will be earned, measured and displayed at all times, increasing fluidity throughout the ecosystem. Meritocracy: In recruitment sense and when it comes to skills, experience and education, users will be able to state what they have, but everything will be subject to meritocracy. Particular focus on projects that are completed within the ecosystem, such as building new parts of the matching, UX, paring candidates, finding jobs, facilitating towards the growth of the platform will offer higher merit. Users on the platform could rate your skills and ability to deliver projects, tasks, tests etc… All of which contributing towards the users overall recognized merits. Vote-ocracy: Everyone has a vote within the ecosystem, however, based on Merit and Trust, your vote has a separate weighting, which you can be sure is unbiased as every decision you make in the platform is subject to community scrutiny, so even with a long established record, if you vote for things or actions that goes against the community so many times, this may also play against your ranking. The entire system of gaining trust and merit would of course create a level of Gamification, and being clever, the creator of such an ecosystem would include increased attainable levels, followed by reward everywhere within the platform to encourage the users to develop themselves, in turn, developing everything around them in the ecosystem much quicker. We are already used to bounty programs externally within ICO’s, so why wouldn’t there be an opportunity to create bounties for almost anything inside a fully decentralized blockchain system? As a user, I may notice a bug, or a piece of functionality that I do or do not like, or even that the platform is missing something. If this ecosystem was a recruitment focused one, I may offer tokens to someone who is already in the company I want to work for, in return for their help preparing me for interview, or even making an introduction to someone senior in the company to help me get my foot in the door. The reward system would intertwine with gamification, and levels of achievement would intertwine with Merit and Trust. In short everything within the ecosystem framework is intertwined and that is what creates a balance and Governance. Which brings us to Governance, as without it there is chaos, so when creating a framework that will have the ability to self govern, you have to release it with the right rules of engagement that allow it to do so. As the framework attracts users and the democracy grows within it, it can truly decentralize without one or a few being able to steer it towards an unfair bias. Governance can come in many forms, but clever technology will wrap all the rules of the framework around the governance of any platform. Something like the one we are discussing would mean any bad actors or individuals looking to manipulate the ecosystem would be extinguished by their own behavior automatically. This then builds a race towards gaining trust from the platform as that will be the only way to have any form of power or influence, yet we can be assured that it will always be for good, as the majority vote, will remain supreme. This also brings me on to the type of blockchain we would run for a framework and ecosystem like this? If we have a system that has verified people and companies over and over, with a long traceable record of trust on the blockchain, what stops these individuals or entities becoming nodes within the ecosystems own blockchain? Users with high trust could act a proof of authority nodes within such an ecosystem and then we have achieved complete self-sufficiency? This is my dream. In my normal day-to-day job, I hold the title chief visionary officer and I spend a good deal of time thinking about: If we could do it different and better, what would that look like? In much of what I have spoken about here, we are applying to the future of my organization, with some of the brightest minds trying to figure out the revolution. So let’s take a step back from what we have discussed. We are talking about creating silo’s of ecosystems and maybe even one day their just being a ‘Super Ecosystem” where everyone’s lives are carried out, no ‘one’ person is in charge but a system that allows us to vote, get credit, earn reward and input as much or as little to the platform where we all host our lives. It sounds far-fetched and even a bit sci-fi, but I can assure you, it is not, now that blockchain is here, the entire world is rushing towards this future and it’s heavily funded with ICO money. At my business Job.com, we have possibly the most unusual agenda for our business, which is how do we remove ourselves completely out of the process? As in a decentralized world, there is no room for recruiters, job boards or even Job.com. We are working towards creating a self serving hiring platform, called a recruitment community ecosystem that will act as the global hirer for everyone and provide the global gig economy a way of earning another income by supporting this framework. Our ecosystem will become part of the daily diet of multiple interactions, efforts, collaborations and value exchanges that we all do, yet now, we will all be rewarded for it. Strangely it excites me as when I look at large centralized companies, they have been exchanging my data, my value in return for profit for so long and I have never had a cent for it. This new world, which has no place for my business or many other centralized models will see a redistribution of power and value exchange between everyone. We have been exchanging value with one another for so long already, with volunteers on Wikipedia, to the endorsement I gave on LinkedIn. What I speak of is nothing new, its already happening, but now blockchain is taking it to another level. I’m confident that with the right frameworks, the ecosystem I have described and far more advanced versions will exist in the near future. Everyday users will simply shift from a centralized to decentralized, it probably won’t feel massively different to the average person, other than once upon a time we did everything for platforms and received nothing, completely at the mercy of their decisions, whereas in the future, the spoils of creating the perfect platforms will be shared amongst the users and its actions will be dictated by the users. For me personally, that is a future I am very much looking forward too. I will end this article with this, I cannot guarantee everything I have said will come true, and I cannot guarantee everything is right, but… I am not afraid to be wrong, I am afraid to do nothing.

Stories, News & Mentions about OCRA

Ocra’s Stories on HackerNoon

Ocra's News & Mentions