A new device created by Ontario researchers is like adding a new, safe front porch to honeybee hives. ProtectaBEE is a honeybee hive entrance that can help protect the pollinators from pests and ...

Bees can now get their hives renovated — and it could save their lives

Netflix series, Beckham, shows David Beckham using the Flow Hive, invented by father and son duo Cedar

Netflix series Beckham opens with David Beckham using a Flow Hive, invented near Lismore

A humble invention, dreamed up in the back shed of a rural property has had a starring role in the opening scenes of Netflix's new hit show, Beckham.

Blockchain stocks are still popular despite Blockchain and decentralization being terms thrown around quite loosely these days. There are numerous reasons behind the popularity of the concepts. For one,

7 Blockchain Stocks to Buy Before Decentralization Goes Mainstream

During the spring and summer when the bees are flying around, that is one of a beekeepers busiest times. For those that create beehives and gather the honey, it is a process treatment, luck and trusting the bees to do their job.

Buzzing business — Beekeeping takes planning and a bit of luck

Other Senior Citizens’ Month activities supported by the Flow Foundation included a Tech Talk, where 120 seniors in Gregory Park, St. Catherine, received valuable insights on the advantages of ...

Honouring senior citizens: Flow Foundation, NCSC celebrate the elderly

Ribbon Cutting: Flow’s Country Manager Sharon Jemmot cuts ribbon to officially launch the SOW Flow Dominica rolled out its latest innovation on Friday with the launch of ‘Store on Wheels’. This new initiative is being touted as part of the company ...

BUSINESS BYTE: Flow rolls out its new Store on Wheels

The 13.4-inch, 1920×1200 IPS display is considerably faster than your typical Windows ultraportable at 120Hz, and the thicker body allows the Flow 13 to pack both USB- and USB-C ports, plus a ...

Asus’ radical ROG Flow Z13 gaming tablet is 46% off

And that future is sooner than you think, he says. Kilic is the president and CEO of HIVE Digital Technologies Ltd., a Vancouver-based digital infrastructure firm. HIVE got its start in 2017 as HIVE Blockchain Technologies—the world’s first publicly ...

HIVE Digital Technologies is Building Green Infrastructure for the AI Revolution

The ACT is the only jurisdiction to have supported a Voice to Parliament so far, with just over half of its votes counted.

ACT only jurisdiction to vote 'yes' on Voice to Parliament, as counting continues

Flow Hive

flowhive

Tailwind CSS is a framework that you’ve probably seen around quite often now and it looks like the technology is being adopted by more and more web developers. What’s unique about this framework is that it uses utility classes instead of OOCSS or BEM and instead of a CSS preprocessing language it relies on PostCSS for transforming CSS. Although we’ve seen quite a spectacular growth for Tailwind CSS this year (433% increase in weekly downloads based on NPM Trends), one thing that’s missing is a set of UI components that can be used to speed up the development process. In this article, I want to introduce you to one of the most popular component libraries based on Tailwind CSS and it’s called Flowbite. It’s an under the MIT license. open-source project available on Github I’ll show you how to get started by installing Flowbite as a plugin for Tailwind CSS and show you some of the components and features that it has. What is Tailwind CSS? is a utility-first framework created by Adam Wathan which redefines the way you interact with stylesheets by moving the CSS logic right into your HTML by using utility classes for spacing, sizing, colors, fonts, and more. Tailwind CSS With the release of the JIT mode the compilation and the number of utility classes that can be used when developing a project increased significantly. What is Flowbite? is a set of UI components built with the utility classes from Tailwind CSS including buttons, dropdowns, modals, tabs, navbars, and more. It can be installed via NPM and included as a plugin inside a new or existing Tailwind CSS project. Flowbite Getting started The first thing you need to do is have a working Tailwind CSS project installed locally. If you haven’t done that yet, I recommend you to check out the on the documentation. installation guide After you’ve done that, you can follow the and learn how to download and install the project. quickstart guide from Flowbite You should first install the Flowbite package via NPM: npm i flowbite Then require the plugin inside the file: tailwind.config.js module.exports = {

    plugins: [
        require('flowbite/plugin')
    ]

} Finally, make sure that the JavaScript file is included just before the closure of your tag: <body <script src="../path/to/flowbite/dist/flowbite.js"></script> Flowbite works with both Tailwind CSS v2.x and v3.x versions. Tailwind CSS components within Flowbite Now that you have a working Tailwind CSS project with Flowbite installed as a plugin you can now start exploring the from the library. Tailwind CSS components The full list of components are as follows: Alerts Badge Breadcrumbs Buttons Button group Cards Dropdowns Forms List groups Typography Modals Tabs Navbar Pagination Progress bar Tables Tooltips Datepicker Here’s an example of a couple of gradient background : buttons from the library <button type="button" class="text-white bg-gradient-to-br from-purple-600 to-blue-500 hover:bg-gradient-to-bl focus:ring-4 focus:ring-blue-300 dark:focus:ring-blue-800 font-medium rounded-lg text-sm px-5 py-2.5 text-center mr-2 mb-2">Purple to Blue</button>
<button type="button" class="text-white bg-gradient-to-r from-cyan-500 to-blue-500 hover:bg-gradient-to-bl focus:ring-4 focus:ring-cyan-300 dark:focus:ring-cyan-800 font-medium rounded-lg text-sm px-5 py-2.5 text-center mr-2 mb-2">Cyan to Blue</button>
<button type="button" class="text-white bg-gradient-to-br from-green-400 to-blue-600 hover:bg-gradient-to-bl focus:ring-4 focus:ring-green-200 dark:focus:ring-green-800 font-medium rounded-lg text-sm px-5 py-2.5 text-center mr-2 mb-2">Green to Blue</button>
<button type="button" class="text-white bg-gradient-to-r from-purple-500 to-pink-500 hover:bg-gradient-to-l focus:ring-4 focus:ring-purple-200 dark:focus:ring-purple-800 font-medium rounded-lg text-sm px-5 py-2.5 text-center mr-2 mb-2">Purple to Pink</button>
<button type="button" class="text-white bg-gradient-to-br from-pink-500 to-orange-400 hover:bg-gradient-to-bl focus:ring-4 focus:ring-pink-200 dark:focus:ring-pink-800 font-medium rounded-lg text-sm px-5 py-2.5 text-center mr-2 mb-2">Pink to Orange</button>
<button type="button" class="text-gray-900 bg-gradient-to-r from-teal-200 to-lime-200 hover:bg-gradient-to-l hover:from-teal-200 hover:to-lime-200 focus:ring-4 focus:ring-lime-200 dark:focus:ring-teal-700 font-medium rounded-lg text-sm px-5 py-2.5 text-center mr-2 mb-2">Teal to Lime</button>
<button type="button" class="text-gray-900 bg-gradient-to-r from-red-200 via-red-300 to-yellow-200 hover:bg-gradient-to-bl focus:ring-4 focus:ring-red-100 dark:focus:ring-red-400 font-medium rounded-lg text-sm px-5 py-2.5 text-center mr-2 mb-2">Red to Yellow</button> As you can see, all of the classes from these elements are based on the Tailwind CSS utility classes and there’s also dark mode support included. Another example would be the dropdown elements which also requires the JavaScript file to be included: <button id="dropdownButton" data-dropdown-toggle="dropdown" class="text-white bg-blue-700 hover:bg-blue-800 focus:ring-4 focus:ring-blue-300 font-medium rounded-lg text-sm px-4 py-2.5 text-center inline-flex items-center dark:bg-blue-600 dark:hover:bg-blue-700 dark:focus:ring-blue-800" type="button">Dropdown button <svg class="ml-2 w-4 h-4" fill="none" stroke="currentColor" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 9l-7 7-7-7"></path></svg></button>

<div id="dropdown" class="hidden z-10 w-44 text-base list-none bg-white rounded divide-y divide-gray-100 shadow dark:bg-gray-700">
  <ul class="py-1" aria-labelledby="dropdownButton">
    <li>
      <a href="#" class="block py-2 px-4 text-sm text-gray-700 hover:bg-gray-100 dark:hover:bg-gray-600 dark:text-gray-200 dark:hover:text-white">Dashboard</a>
     </li>
    <li>
      <a href="#" class="block py-2 px-4 text-sm text-gray-700 hover:bg-gray-100 dark:hover:bg-gray-600 dark:text-gray-200 dark:hover:text-white">Settings</a>
    </li>
    <li>
      <a href="#" class="block py-2 px-4 text-sm text-gray-700 hover:bg-gray-100 dark:hover:bg-gray-600 dark:text-gray-200 dark:hover:text-white">Earnings</a>
    </li>
    <li>
      <a href="#" class="block py-2 px-4 text-sm text-gray-700 hover:bg-gray-100 dark:hover:bg-gray-600 dark:text-gray-200 dark:hover:text-white">Sign out</a>
    </li>
  </ul>
</div> For the , you can use the data attribute to connect the menu with the button using the event listeners from Flowbite. dropdown component data-dropdown-toggle=”dropdownId” I’ll show you one more example from the section: form components <form>
  <div class="mb-6">
    <label for="email" class="block mb-2 text-sm font-medium text-gray-900 dark:text-gray-300">Your email</label>
    <input type="email" id="email" class="bg-gray-50 border border-gray-300 text-gray-900 text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5 dark:bg-gray-700 dark:border-gray-600 dark:placeholder-gray-400 dark:text-white dark:focus:ring-blue-500 dark:focus:border-blue-500" placeholder="name@flowbite.com" required="">
  </div>
  <div class="mb-6">
      <label for="password" class="block mb-2 text-sm font-medium text-gray-900 dark:text-gray-300">Your password</label>
      <input type="password" id="password" class="bg-gray-50 border border-gray-300 text-gray-900 text-sm rounded-lg focus:ring-blue-500 focus:border-blue-500 block w-full p-2.5 dark:bg-gray-700 dark:border-gray-600 dark:placeholder-gray-400 dark:text-white dark:focus:ring-blue-500 dark:focus:border-blue-500" required="">
  </div>
  <div class="flex items-start mb-6">
    <div class="flex items-center h-5">
      <input id="remember" aria-describedby="remember" type="checkbox" class="w-4 h-4 bg-gray-50 rounded border border-gray-300 focus:ring-3 focus:ring-blue-300 dark:bg-gray-700 dark:border-gray-600 dark:focus:ring-blue-600 dark:ring-offset-gray-800" required="">
    </div>
    <div class="ml-3 text-sm">
      <label for="remember" class="font-medium text-gray-900 dark:text-gray-300">Remember me</label>
    </div>
  </div>
  <button type="submit" class="text-white bg-blue-700 hover:bg-blue-800 focus:ring-4 focus:ring-blue-300 font-medium rounded-lg text-sm w-full sm:w-auto px-5 py-2.5 text-center dark:bg-blue-600 dark:hover:bg-blue-700 dark:focus:ring-blue-800">Submit</button>
</form> The form components include custom styles and variants for text inputs, checkboxes, radio buttons, toggle buttons, file uploads, and many more. How to use dark mode All of the components from Flowbite already support dark mode and since the release of Tailwind CSS v3.x, these styles will be automatically applied based on your browser preference. If you want to create a dark mode switcher, you can check out the official guide on Flowbite on how to integrate a . dark mode switcher with Tailwind CSS Figma design files If you’re a fan of the Figma software design tool, you’ll be happy to learn that Flowbite also has a included based on the utility classes from Tailwind CSS. Figma design file You can use this design file to prototype and design your components and pages before coding them with Tailwind CSS and Flowbite.

How to Get Started with Flowbite, an Open Source Tailwind CSS component Library

Cómo comenzar con Flowbite, una biblioteca de componentes CSS de Tailwind de código abierto

Become a Magician: Consumer Brain Devices 2024

Comparing Best Data Table Options for Svelte Projects in 2024

An Architect’s Guide to the Top 10 Tools Needed to Build the Modern Data Lake

আধুনিক ডেটা লেক তৈরির জন্য প্রয়োজনীয় শীর্ষ 10টি সরঞ্জামের জন্য একজন স্থপতির গাইড

Ein Architektenhandbuch zu den 10 wichtigsten Tools zum Aufbau eines modernen Data Lake

Guía para arquitectos sobre las 10 herramientas principales necesarias para crear un lago de datos moderno

Guide de l’architecte sur les 10 principaux outils nécessaires à la création du lac de données moderne

आधुनिक डेटा लेक के निर्माण के लिए आवश्यक शीर्ष 10 उपकरणों के लिए एक आर्किटेक्ट गाइड

現代のデータレイクを構築するために必要なトップ 10 ツールに関するアーキテクト ガイド

Um guia de arquiteto para as 10 principais ferramentas necessárias para construir o moderno Data Lake

Руководство архитектора по 10 лучшим инструментам, необходимым для создания современного озера данных

Modern Veri Gölünü Oluşturmak İçin Gereken En İyi 10 Araca Yönelik Bir Mimarın Rehberi

Hướng dẫn của Kiến trúc sư về 10 Công cụ hàng đầu cần thiết để Xây dựng Hồ dữ liệu hiện đại

架构师指南：构建现代数据湖所需的十大工具

Is Your Apache Ni-Fi Ready for Production?

Apache NiFi is a powerful tool for automating data flows with a large number of built-in processors. However, sometimes there is a need for specific processors to handle unique requirements and data stores. In such cases, creating custom processors becomes a necessity. In this article, we will go through the steps to create a very simple custom processor for Apache NiFi. Setup and sources gradle setup dependencies {
	compile "org.apache.nifi:nifi-api:*"
	compile "org.apache.nifi:nifi-utils:1.9.2"
	testCompile "org.apache.nifi:nifi-mock:1.9.2"
} source of the custom processor import org.apache.nifi.annotation.behavior.ReadsAttribute;
import org.apache.nifi.annotation.behavior.ReadsAttributes;
import org.apache.nifi.annotation.behavior.WritesAttribute;
import org.apache.nifi.annotation.behavior.WritesAttributes;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.*;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.util.StandardValidators;

import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
import java.util.*;

@Tags({"example"})
@CapabilityDescription("Hello World to output")
@SeeAlso({})
@ReadsAttributes({@ReadsAttribute(attribute = "", description = "")})
@WritesAttributes({@WritesAttribute(attribute = "", description = "")})
public class MyProcessor extends AbstractProcessor {

    public static final PropertyDescriptor propertyDescriptor = new PropertyDescriptor
            .Builder().name("name")
            .displayName("name")
            .description("Name to print")
            .required(true)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
            .build();

    public static final Relationship successRelations = new Relationship.Builder()
            .name("success")
            .description("If everything is ok")
            .build();

    public static final Relationship failureRelations = new Relationship.Builder()
            .name("failure")
            .description("If something is wrong")
            .build();

    private List<PropertyDescriptor> descriptorList;

    private Set<Relationship> relationshipSet;

    @Override
    protected void init(final ProcessorInitializationContext context) {
        final List<PropertyDescriptor> descriptors = new ArrayList<PropertyDescriptor>();
        descriptors.add(propertyDescriptor);
        this.descriptorList = Collections.unmodifiableList(descriptors);

        final Set<Relationship> relationships = new HashSet<Relationship>();
        relationships.add(successRelations);
        relationships.add(failureRelations);
        this.relationshipSet = Collections.unmodifiableSet(relationships);
    }

    public Set<Relationship> getRelationshipSet() {
        return this.relationshipSet;
    }

    @Override
    public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
        return descriptorList;
    }

    @OnScheduled
    public void onScheduled(final ProcessContext context) {

    }

    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
        FlowFile flowFile = session.get();
        if (flowFile == null) {
            return;
        }

        try {
            String name = context.getProperty(propertyDescriptor).evaluateAttributeExpressions(flowFile).getValue();

            String result = "Hello " + name;
            session.putAttribute(flowFile, "result", result);

            try (OutputStream flowFileOutputStream = session.write(flowFile)) {
                flowFileOutputStream.write(result.getBytes(StandardCharsets.UTF_8));
            }

            session.transfer(flowFile, successRelations);
        } catch (IOException e) {
		        //IO Error processing error log
		        session.transfer(flowFile, failureRelations);
		    } catch (ProcessException e) {
				    // Process error log
		        getLogger().error("Processing error", e);
		        session.transfer(flowFile, failureRelations);
		    }
    }
} Build and deploy Build the project with Gradle: gradle build.
Find the generated .nar file in the target folder of your module.
Copy the .nar file to the lib folder of your NiFi installation. Screenshot of how the custom processor integrates seamlessly I hope this article was useful for you! Creating a custom processor for Apache NiFi allows you to customize it to meet the specific requirements of your project. By following the steps above, you will be able to develop and integrate a custom processor that will enhance the processing capabilities of your system. Apache NiFi is a powerful tool for automating data flows with a large number of built-in processors. However, sometimes there is a need for specific processors to handle unique requirements and data stores. In such cases, creating custom processors becomes a necessity. In this article, we will go through the steps to create a very simple custom processor for Apache NiFi. Setup and sources gradle setup gradle setup dependencies {
	compile "org.apache.nifi:nifi-api:*"
	compile "org.apache.nifi:nifi-utils:1.9.2"
	testCompile "org.apache.nifi:nifi-mock:1.9.2"
} dependencies {
	compile "org.apache.nifi:nifi-api:*"
	compile "org.apache.nifi:nifi-utils:1.9.2"
	testCompile "org.apache.nifi:nifi-mock:1.9.2"
} source of the custom processor source of the custom processor import org.apache.nifi.annotation.behavior.ReadsAttribute;
import org.apache.nifi.annotation.behavior.ReadsAttributes;
import org.apache.nifi.annotation.behavior.WritesAttribute;
import org.apache.nifi.annotation.behavior.WritesAttributes;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.*;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.util.StandardValidators;

import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
import java.util.*;

@Tags({"example"})
@CapabilityDescription("Hello World to output")
@SeeAlso({})
@ReadsAttributes({@ReadsAttribute(attribute = "", description = "")})
@WritesAttributes({@WritesAttribute(attribute = "", description = "")})
public class MyProcessor extends AbstractProcessor {

    public static final PropertyDescriptor propertyDescriptor = new PropertyDescriptor
            .Builder().name("name")
            .displayName("name")
            .description("Name to print")
            .required(true)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
            .build();

    public static final Relationship successRelations = new Relationship.Builder()
            .name("success")
            .description("If everything is ok")
            .build();

    public static final Relationship failureRelations = new Relationship.Builder()
            .name("failure")
            .description("If something is wrong")
            .build();

    private List<PropertyDescriptor> descriptorList;

    private Set<Relationship> relationshipSet;

    @Override
    protected void init(final ProcessorInitializationContext context) {
        final List<PropertyDescriptor> descriptors = new ArrayList<PropertyDescriptor>();
        descriptors.add(propertyDescriptor);
        this.descriptorList = Collections.unmodifiableList(descriptors);

        final Set<Relationship> relationships = new HashSet<Relationship>();
        relationships.add(successRelations);
        relationships.add(failureRelations);
        this.relationshipSet = Collections.unmodifiableSet(relationships);
    }

    public Set<Relationship> getRelationshipSet() {
        return this.relationshipSet;
    }

    @Override
    public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
        return descriptorList;
    }

    @OnScheduled
    public void onScheduled(final ProcessContext context) {

    }

    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
        FlowFile flowFile = session.get();
        if (flowFile == null) {
            return;
        }

        try {
            String name = context.getProperty(propertyDescriptor).evaluateAttributeExpressions(flowFile).getValue();

            String result = "Hello " + name;
            session.putAttribute(flowFile, "result", result);

            try (OutputStream flowFileOutputStream = session.write(flowFile)) {
                flowFileOutputStream.write(result.getBytes(StandardCharsets.UTF_8));
            }

            session.transfer(flowFile, successRelations);
        } catch (IOException e) {
		        //IO Error processing error log
		        session.transfer(flowFile, failureRelations);
		    } catch (ProcessException e) {
				    // Process error log
		        getLogger().error("Processing error", e);
		        session.transfer(flowFile, failureRelations);
		    }
    }
} import org.apache.nifi.annotation.behavior.ReadsAttribute;
import org.apache.nifi.annotation.behavior.ReadsAttributes;
import org.apache.nifi.annotation.behavior.WritesAttribute;
import org.apache.nifi.annotation.behavior.WritesAttributes;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
import org.apache.nifi.annotation.documentation.SeeAlso;
import org.apache.nifi.annotation.documentation.Tags;
import org.apache.nifi.annotation.lifecycle.OnScheduled;
import org.apache.nifi.components.PropertyDescriptor;
import org.apache.nifi.expression.ExpressionLanguageScope;
import org.apache.nifi.flowfile.FlowFile;
import org.apache.nifi.processor.*;
import org.apache.nifi.processor.exception.ProcessException;
import org.apache.nifi.processor.util.StandardValidators;

import java.io.OutputStream;
import java.nio.charset.StandardCharsets;
import java.util.*;

@Tags({"example"})
@CapabilityDescription("Hello World to output")
@SeeAlso({})
@ReadsAttributes({@ReadsAttribute(attribute = "", description = "")})
@WritesAttributes({@WritesAttribute(attribute = "", description = "")})
public class MyProcessor extends AbstractProcessor {

    public static final PropertyDescriptor propertyDescriptor = new PropertyDescriptor
            .Builder().name("name")
            .displayName("name")
            .description("Name to print")
            .required(true)
            .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
            .expressionLanguageSupported(ExpressionLanguageScope.FLOWFILE_ATTRIBUTES)
            .build();

    public static final Relationship successRelations = new Relationship.Builder()
            .name("success")
            .description("If everything is ok")
            .build();

    public static final Relationship failureRelations = new Relationship.Builder()
            .name("failure")
            .description("If something is wrong")
            .build();

    private List<PropertyDescriptor> descriptorList;

    private Set<Relationship> relationshipSet;

    @Override
    protected void init(final ProcessorInitializationContext context) {
        final List<PropertyDescriptor> descriptors = new ArrayList<PropertyDescriptor>();
        descriptors.add(propertyDescriptor);
        this.descriptorList = Collections.unmodifiableList(descriptors);

        final Set<Relationship> relationships = new HashSet<Relationship>();
        relationships.add(successRelations);
        relationships.add(failureRelations);
        this.relationshipSet = Collections.unmodifiableSet(relationships);
    }

    public Set<Relationship> getRelationshipSet() {
        return this.relationshipSet;
    }

    @Override
    public final List<PropertyDescriptor> getSupportedPropertyDescriptors() {
        return descriptorList;
    }

    @OnScheduled
    public void onScheduled(final ProcessContext context) {

    }

    @Override
    public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
        FlowFile flowFile = session.get();
        if (flowFile == null) {
            return;
        }

        try {
            String name = context.getProperty(propertyDescriptor).evaluateAttributeExpressions(flowFile).getValue();

            String result = "Hello " + name;
            session.putAttribute(flowFile, "result", result);

            try (OutputStream flowFileOutputStream = session.write(flowFile)) {
                flowFileOutputStream.write(result.getBytes(StandardCharsets.UTF_8));
            }

            session.transfer(flowFile, successRelations);
        } catch (IOException e) {
		        //IO Error processing error log
		        session.transfer(flowFile, failureRelations);
		    } catch (ProcessException e) {
				    // Process error log
		        getLogger().error("Processing error", e);
		        session.transfer(flowFile, failureRelations);
		    }
    }
} Build and deploy Build and deploy Build the project with Gradle: gradle build. Find the generated .nar file in the target folder of your module. Copy the .nar file to the lib folder of your NiFi installation. Build the project with Gradle: gradle build . gradle build Find the generated .nar file in the target folder of your module. .nar Copy the .nar file to the lib folder of your NiFi installation. .nar lib Screenshot of how the custom processor integrates seamlessly Screenshot of how the custom processor integrates seamlessly I hope this article was useful for you! Creating a custom processor for Apache NiFi allows you to customize it to meet the specific requirements of your project. By following the steps above, you will be able to develop and integrate a custom processor that will enhance the processing capabilities of your system.

Building a Simple Custom Processor With Apache Ni-Fi

The Open-Source Libraries to Check Out for LLM Building

LLM বিল্ডিংয়ের জন্য ওপেন-সোর্স লাইব্রেরি চেক আউট

Die Open-Source-Bibliotheken, die Sie für den LLM-Aufbau ausprobieren sollten

Las bibliotecas de código abierto que debe consultar para la construcción de LLM

Les bibliothèques open source à consulter pour la construction LLM

एलएलएम निर्माण के लिए जाँचने योग्य ओपन-सोर्स लाइब्रेरीज़

LLM 構築のためにチェックすべきオープンソース ライブラリ

LLM 구축을 위해 체크아웃할 오픈 소스 라이브러리

As bibliotecas de código aberto para verificar o edifício LLM

Библиотеки с открытым исходным кодом, которые стоит проверить для построения LLM

Yüksek Lisans Binası İçin İncelenecek Açık Kaynak Kütüphaneleri

Các thư viện nguồn mở để kiểm tra Tòa nhà LLM

构建 LLM 时需要检查的开源库

Setting up Data Streams Using NiPyAPI: UI-Free Guide

306 Stories To Learn About Webdev

If you have a lot of private enterprise data, how can you use a ChatGPT-like AI system to help you search for relevant information faster? Or if you have lots of personal information, how can you find old memories without the exact keywords? In this practical guide, I will show you 5 simple steps to implement semantic search with the help of LangChain, vector databases, and large language models. Best of all, I will use all open-source components that can be run locally on your own machine. No need for any cloud SaaS or API keys, and your data will never leave your office or home. Use Case Analysis Our has a based forum that has been running for over 5 years. There is a wealth of information, but sometimes, it is hard for newcomers to find the right answers to their questions quickly and succinctly. community Discourse There are a few reasons, which are very common for Google/pagerank based search solutions: Search is based on keywords, and often, users do not know the right terms or exact wording. Even if the user figured out the best search terms for their particular question, the return result is a list of top discussion threads. And the answer to the question might not be in the first few posts or could be buried deep in a prolonged discussion thread. In general, search engines do not understand long sentences very well if you ask a long-winded question or two questions in one sentence. You might ask another question, “ChatGPT v4 is already so good, why don’t you ask questions directly from chatGPT?” Again, there are several strong reasons: ChatGPT (v3.5 or v4) model was trained on data before November 2021, thus, it does not have the latest information since then. The latest OpenAI website retrieval plug-in can search over the website for the latest info, but it is still quite slow and often stuck. It is neither efficient nor cost-effective for your enterprise's internal use. You don’t want OpenAI and Microsoft to access your proprietary enterprise data or your personal data. So, my goal is that if someone comes in and asks, “What is the easiest way to run an NKN node?”, he/she should be able to get a straight answer in a few sentences. I use this public NKN forum only as an example to show you the typical dataset and typical questions and answers for a special enterprise knowledge base. Design Approach After some research and experiments, I narrowed down my plan into 5 steps: Do a quick Proof of Concept using cloud service and API Clean and prep my data Convert my entire data set into a vector database Use vector database to find the best matches to my query Use LLM to distill the best answer to the original question If you want to pause here and learn some basics about LangChain, vector databases, large language models, and how they work together, I will recommend and . And below, is a short introduction to these concepts. this video this video Key Components : LangChain is a framework for developing applications powered by language models. It is the glue or middleware that chains all the different components together to accomplish complex tasks semi-autonomously. Langchain The most powerful and differentiated applications will not only call out to a language model via an API but will also: Be data-aware: Connect a language model to other sources of data. Be agentic: Allow a language model to interact with its environment. : A vector database is a specialized storage and retrieval system designed to handle high-dimensional vector data efficiently and is often used in machine learning, artificial intelligence, and data analytics applications. Vector database These databases enable users to perform similarity searches or nearest neighbor searches, allowing them to find the most similar items within the database based on their vector representations. Pinecone is one of the leading commercial vector databases. There are open-source alternatives such as FAISS, Chroma, and Supabase. : A large language model like GPT (Generative Pre-trained Transformer) is an advanced artificial intelligence model that utilizes deep learning techniques to process and generate human-like text. Large Language Model With billions of parameters, GPT-3/4 is trained on massive amounts of data from diverse sources, allowing it to understand context, semantics, and syntax. As a result, GPT-3/4 can perform various natural language processing tasks, such as text generation, translation, summarization, and question-answering, with remarkable accuracy and fluency. There are several community-driven LLM models based on Meta’s LLaMA, e.g., Alpaca, Vicuna, Koala, and GPT4All, which can be run locally on a laptop (read ). my tutorial here Step 1: A Proof of Concept Before you spend more effort on your custom solution, let’s first get familiar with the key framework LangChain and the components in a playground. Thanks to GUI-based , you can now play with this entire solution without any code. flowise.ai This step is optional, and you can skip to step 2 if you want to go straight into coding locally or if you don’t have API keys for OpenAI or Pinecone. Prerequisites: You obtain the and in advance. Both are useful not only for this exercise but also for other AI-related experiments. OpenAI API key Pinecone API key In Pinecone, you need to create a new index called “ ” with dimension . Note there is an environment name auto-generated by Pinecone to designate where this index instance is run: “ ” You can use the default metrics of “ ” test 1536 us-west1-gcp-free. Cosine. A small dataset in text file format (let’s say ): it could be as simple as copy-paste a few paragraphs from a Word or PDF document. In addition, for privacy reasons, you want to use a very small data set and preferably publicly available information. Since both APIs are cloud-based. test.txt Now. head to , download the code from their , and install and run locally (you need to have Node Javascript installed). You can access it from a browser: . Flowise AI Github http://localhost:3000/marketplaces And from Marketplace, you can choose a template called “ ”, and use this template. The UI is self-explanatory: you fill in the OpenAI API key twice (one for embedding and one for LLM), the Pinecone API key, index name, environment, and metrics. Conversational QA Retrieval Chain And upload the data set text file. It might take a couple of minutes depending on the size of your text film. Then you click the little chat bubble on the top right corner, and you can start to ask any questions! So this will give you a visual understanding of the data processing pipeline of our forum use case, and help you implement your own solution by replacing all the cloud API with the local service using open-source software. LangChain will stay put since it is an open-source software and the master glue we need. Large language model: -> OpenAI LLaMA/Alpaca Vector database: -> Pinecone FAISS But before we do that, we first have to clean up and prepare your enterprise data for ingestion into the vector database. Step 2: Clean and Prep Data The NKN forum uses Discourse software, a popular open-source discussion and community forum solution. It uses the Postgres SQL database as its core data store, which has two tables that are especially relevant for our Q&A retrieval use case topics posts. For example, the database schema of “ ” is listed partially below. The most relevant columns for us are: “ ” (matching to the topics table), “ ” (original post content in the text without formatting), and “ ” (how popular or well-read is this post). posts topic_id raw score It is quite natural to map “ ” to questions (or “ ”), and “ ” to answers (“ ”). The terms in parentheses are for GPT-like LLMs. There are many implementation considerations and optimizations here, which would suffice for another article. topics prompts posts::raw completion But I would like to keep it simple and choose only the best post out of the entire thread to match the original topic. So first of all, we need to get both tables out of the database and into JSON files. To do that, I did a DB dump in Discourse, and then import it into a standalone Postgres SQL database running on my Macbook Pro M1. From the standalone Postgres SQL, I can export JSON files for the two tables: and . posts topics Then I wrote some simple Python script to do the following: For each topic, use SQL to select the posts with the highest “score.” The resulting “topic” and “post::raw” become one “prompt” and “completion” pair. Save the above into a JSON file called training.json (you can use it for many other training purposes, e.g., finetuning an LLM model, using GPT-3’s finetuning API, etc.) Write into a text file called training.txt: one line for prompt, and one line for completion, and separate each pair with two new lines. Now your file should look a bit like this: Step 3: Convert My Entire Data Set Into a Vector Database I pasted the Python script below, which is mostly self-explanatory. A few notes: You need to download or find the weights for your local LLM model. In my case, I used Alpaca 7B parameter 4-bit quantized version. You can find them on huggingface. There are many similar ones that are fine-tuned based on Meta’s LLaMA 7B/13B models, e.g., Alpaca, Vicuna, Koala, GPT4All. You can use any of them. Optionally, you have followed and installed Llama.cpp locally and successfully ran Llama.cpp with the weights from above. So you know the weights are good. my instruction The text splitter will split your long text file into chunks, each having a max length of . 1,000 Then, this chunk of text is fed into the LLM for embedding, which basically tokenizes the words, and converts them into vectors based on their meaning. Each chunk, after tokenization, might become up to 1,400 tokens. Since on average, 3 words create about 4 tokens. This is also the reason I have (default is 512) in order to have a large enough context window for embedding. n_ctx = 2048 Finally, the vectors are inserted into the FAISS vector database and indexed. The database is saved as a local file in a directory named “faiss_index”. from langchain.embeddings import LlamaCppEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import FAISS
from langchain.document_loaders import TextLoader

embeddings = LlamaCppEmbeddings(model_path="./ggml-alpaca-7b-q4.bin", n_ctx= 2048)

from langchain.document_loaders import TextLoader
loader = TextLoader('./training_full_clean.txt')
documents = loader.load()
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size = 1000, 
    chunk_overlap = 50,
    length_function = len)
docs = text_splitter.split_documents(documents)

db = FAISS.from_documents(docs, embeddings)
db.save_local("faiss_index")

# Now let's test it out
query = "Who is Whitfield Diffie?"
docs = db.similarity_search(query)
for doc in docs:
    print(doc.page_content) I highly recommend you start with a very small dataset text file to test first. If your dataset is large, this will take a long time. My locally run Llama.cpp can embed about 20 tokens per second, so each chunk of 1,000 text words could take about 1 minute to do embedding. My vector DB has about 7,500 entries, and it will take me about 100 hours on a Macbook Pro M1. The good news is that you only need to do this once. [Author note: please see comments to use different embedding module such as huggingface for much faster performance on large datasets] Step 4: Use a Vector Database to Find the Best Matches to My Query Again, I provide some simple Python code to do the query and summarization. Since we use a template chain called “ ”, some details might not be obvious. But under the hood, it involved the following steps: RetrievalQA When a user types in a query, the large language model converts the text into embedding and a vector (basically an array of floating point numbers) representing the query. The vector database then does a similarity search to find the candidate result vectors that are closest related to the query vector. Now, we return those candidate result vectors, which are already encoded in the way that LLM understands, to LLM for final processing. from langchain.vectorstores import FAISS
from langchain.llms import LlamaCpp
from langchain.embeddings import LlamaCppEmbeddings
from langchain.callbacks.base import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.chains import RetrievalQA

# Load DB from local file
embeddings = LlamaCppEmbeddings(model_path="./ggml-alpaca-7b-q4.bin")
new_db = FAISS.load_local("faiss_index", embeddings)

# Use llama-cpp as the LLM for langchain
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])
llm = LlamaCpp(
    model_path="./ggml-alpaca-7b-q4.bin",
    n_ctx= 2048,
    callback_manager=callback_manager, 
    verbose=True,
    use_mlock=True
)

retriever = new_db.as_retriever()

# Conversational QA retrieval chain
qa = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever)

while True:
    user_input = input("You: ")
    if user_input.lower() == "exit":
        break

    response = qa.run(user_input)
    print(f"AI: {response}") Step 5: Use LLM to Distill the Best Answer to the Original Question In the final step and again under the hood, the “ ” template automatically constructs a special , which includes the following: RetrieverQA prompt : Instruct the LLM what kind of task it needs to do, and in this case, “ ” System message Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. : Add the best candidate result vectors as context, as well as the query vector at the end. User message Once LLM receives such a prompt, it will try its best to sift through the candidate results and find the best answer to the original question. It summarizes, rephrases, and even finds separate responses to two separate questions in the original query (e.g., “Who is Stephen Wolfram, and how is he related to NKN?”). Sometimes, we can be quite amazed at how capable LLMs have become to handle this type of Q&A task. What Next? Now, you are well on your way to your office conference room to show off your new demo! And what can you do next? Congratulations, you have just successfully built your first semantic search tool on your private data using an entirely open-source toolkit! Try alternative components for vector databases or large language models that fit your data’s needs best. Optimize, scale, and productize your semantic search solution for your entire company. Dig deeper into LangChain and its infinite capabilities to solve other pressing work or life-related problems. Maybe you can even contribute your own chain as a template for other people to use. Happy hacking! 🙂

Stories, News & Mentions about FLOWHIVE

Flowhive’s Stories on HackerNoon

Flowhive's News & Mentions